I think the general advice is to do a full re-index on a major version
upgrade. Also - did you ever commit?
On Sun, Jan 10, 2021 at 11:13 AM Flowerday, Matthew J <
matthew.flower...@gb.unisys.com> wrote:
> Hi There
>
>
>
> Thanks for contacting me.
>
>
>
> I carried out this analysis of the solr log from the updates I carried out
> at the time:
>
>
>
> Looking at the update requests sent to Solr. The first update of an
> existing record generated
>
>
>
> 2021-01-07 06:04:18.958 INFO (qtp1458091526-17) [ x:uleaf]
> o.a.s.u.p.LogUpdateProcessorFactory [uleaf] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[9901020319M01-X11
> (1688206792619720704)]} 0 59
>
> 2021-01-07 06:04:19.186 INFO
> (searcherExecutor-15-thread-1-processing-x:uleaf) [ x:uleaf]
> o.a.s.c.QuerySenderListener QuerySenderListener done.
>
> 2021-01-07 06:04:19.196 INFO
> (searcherExecutor-15-thread-1-processing-x:uleaf) [ x:uleaf]
> o.a.s.c.SolrCore [uleaf] Registered new searcher autowarm time: 1 ms
>
> 2021-01-07 06:04:19.198 INFO (qtp1458091526-23) [ x:uleaf]
> o.a.s.u.p.LogUpdateProcessorFactory [uleaf] webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
> 0 228
>
>
>
> And the record was duplicated:
>
>
>
>
>
> The next update generated
>
>
>
> 2021-01-07 06:10:59.786 INFO (qtp1458091526-17) [ x:uleaf]
> o.a.s.u.p.LogUpdateProcessorFactory [uleaf] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[9901020319M01-X11
> (1688207212953993216)]} 0 20
>
> 2021-01-07 06:10:59.974 INFO
> (searcherExecutor-15-thread-1-processing-x:uleaf) [ x:uleaf]
> o.a.s.c.QuerySenderListener QuerySenderListener done.
>
> 2021-01-07 06:10:59.982 INFO
> (searcherExecutor-15-thread-1-processing-x:uleaf) [ x:uleaf]
> o.a.s.c.SolrCore [uleaf] Registered new searcher autowarm time: 0 ms
>
> 2021-01-07 06:10:59.998 INFO (qtp1458091526-26) [ x:uleaf]
> o.a.s.u.p.LogUpdateProcessorFactory [uleaf] webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
> 0 208
>
>
>
> Which looks the same as the previous command – so no real difference here.
>
>
>
> And then the records looked like
>
>
>
>
>
> And this shows that the original (7.7.1) item is untouched and only the
> 8.6.3 item is updated on subsequent updates.
>
>
>
> A brand new record being sent to solr generate this dialog
>
>
>
> 2021-01-07 06:20:10.645 INFO (qtp1458091526-25) [ x:uleaf]
> o.a.s.u.p.LogUpdateProcessorFactory [uleaf] webapp=/solr path=/update
> params={wt=javabin&version=2}{add=[9901020319M01-X15 (1688207790576762880),
> 9901020319M01-DI21 (1688207790587248640)]} 0 15
>
> 2021-01-07 06:20:10.798 INFO
> (searcherExecutor-15-thread-1-processing-x:uleaf) [ x:uleaf]
> o.a.s.c.QuerySenderListener QuerySenderListener done.
>
> 2021-01-07 06:20:10.802 INFO
> (searcherExecutor-15-thread-1-processing-x:uleaf) [ x:uleaf]
> o.a.s.c.SolrCore [uleaf] Registered new searcher autowarm time: 0 ms
>
> 2021-01-07 06:20:10.803 INFO (qtp1458091526-23) [ x:uleaf]
> o.a.s.u.p.LogUpdateProcessorFactory [uleaf] webapp=/solr path=/update
> params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=}
> 0 153
>
>
>
> And this has a similar update request line as the others – so no
> differences here. Solr just seems to leave the migrated records as is and
> just creates a duplicate when they are updated for some reason.
>
>
>
> I hope this is what you are after.
>
>
>
> Many Thanks
>
>
>
> Matthew
>
>
>
> *Matthew Flowerday* | Consultant | ULEAF
>
> Unisys | 01908 774830| matthew.flower...@unisys.com
>
> Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17
> 8LX
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
> *From:* xiefengchang <fengchang_fi...@163.com>
> *Sent:* 10 January 2021 08:44
> *To:* solr-user@lucene.apache.org
> *Subject:* Re:Query over migrating a solr database from 7.7.1 to 8.7.0
>
>
>
> *EXTERNAL EMAIL - Be cautious of all links and attachments.*
>
> can you show the update request?
>
>
>
>
>
>
>
>
>
>
>
> At 2021-01-07 20:25:13, "Flowerday, Matthew J" <
> matthew.flower...@gb.unisys.com> wrote:
>
> Hi There
>
>
>
> I have recently upgraded a solr database from 7.7.1 to 8.7.0 and not wiped
> the database and re-indexed (as this would take too long to run on site).
>
>
>
> On my local windows machine I have a single solr server 7.7.1 installation
>
>
>
> I upgraded in the following manner
>
>
>
> - Installed windows solr 8.7.0 on my machine in a different folder
> - Copied the core related folder (holding conf, data, lib,
> core.properties) from 7.7.1 to the new 8.7.0 folder
> - Brought up the solr
> - Checked that queries work through the Solr Admin Tool and our
> application
>
>
>
> This all worked fine until I tried to update a record which had been
> created under 7.7.1. Instead of marking the old record as deleted it
> effectively created a new copy of the record with the change in and left
> the old image as still visible. When I updated the record again it then
> correctly updated the new 8.7.0 version without leaving the old image
> behind. If I created a new record and then updated it the solr record would
> be updated correctly. The issue only seemed to affect the old 7.7.1 created
> records.
>
>
>
> An example of the duplication as follows (the first record is 7.7.1
> created version and the second record is the 8.7.0 version after carrying
> out an update):
>
>
>
> {
>
> "*responseHeader*":{
>
> "*status*":0,
>
> "*QTime*":4,
>
> "*params*":{
>
> "*q*":"id:9901020319M01-N26",
>
> "*_*":"1610016003669"}},
>
> "*response*":{"*numFound*":2,"*start*":0,"*numFoundExact*":true,"*docs*
> ":[
>
> {
>
> "*id*":"9901020319M01-N26",
>
> "*groupId*":"9901020319M01",
>
> "*urn*":"N26",
>
> "*specification*":"nominal",
>
> "*owningGroupId*":"9901020319M01",
>
> "*description*":"N26, Yates, Mike, Alan, Richard, MALE",
>
> "*group_t*":"9901020319M01",
>
> "*nominalUrn_t*":"N26",
>
> "*dateTimeCreated_dtr*":"2020-12-30T12:00:53Z",
>
> "*dateTimeCreated_dt*":"2020-12-30T12:00:53Z",
>
> "*title_t*":"Captain",
>
> "*surname_t*":"Yates",
>
> "*qualifier_t*":"Voyager",
>
> "*forename1_t*":"Mike",
>
> "*forename2_t*":"Alan",
>
> "*forename3_t*":"Richard",
>
> "*sex_t*":"MALE",
>
> "*orderedType_t*":"Nominal",
>
> "*_version_*":1687507566832123904},
>
> {
>
> "*id*":"9901020319M01-N26",
>
> "*groupId*":"9901020319M01",
>
> "*urn*":"N26",
>
> "*specification*":"nominal",
>
> "*owningGroupId*":"9901020319M01",
>
> "*description*":"N26, Yates, Mike, Alan, Richard, MALE",
>
> "*group_t*":"9901020319M01",
>
> "*nominalUrn_t*":"N26",
>
> "*dateTimeCreated_dtr*":"2020-12-30T12:00:53Z",
>
> "*dateTimeCreated_dt*":"2020-12-30T12:00:53Z",
>
> "*title_t*":"Captain",
>
> "*surname_t*":"Yates",
>
> "*qualifier_t*":"Voyager enterprise defiant yorktown xx yy",
>
> "*forename1_t*":"Mike",
>
> "*forename2_t*":"Alan",
>
> "*forename3_t*":"Richard",
>
> "*sex_t*":"MALE",
>
> "*orderedType_t*":"Nominal",
>
> "*_version_*":1688224966566215680}]
>
> }}
>
>
>
> I checked the solrconfig.xml file and it does have a uniqueKey set up
>
>
>
> <field name="id" type="string" indexed="true" stored="true"
> required="true" multiValued="false" />
>
>
>
> <uniqueKey>id</uniqueKey>
>
>
>
> I was wondering if this behaviour is expected and if there is a way to
> make sure that records created under a previous version are updated
> correctly (so that the old data is deleted when updated).
>
>
>
> Also am I upgrading solr correctly as it could be that the way I have
> upgraded it might be causing this issue (I tried hunting through the solr
> documentation online but struggled to find window upgrade notes and the
> above steps I worked out by trial and error).
>
>
>
> Many thanks
>
>
>
> Matthew
>
>
>
> *Matthew Flowerday* | Consultant | ULEAF
>
> Unisys | 01908 774830| matthew.flower...@unisys.com
>
> Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17
> 8LX
>
>
>
> [image: unisys_logo] <http://www.unisys.com/>
>
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
> MATERIAL and is for use only by the intended recipient. If you received
> this in error, please contact the sender and delete the e-mail and its
> attachments from all devices.
>
> [image: Grey_LI] <http://www.linkedin.com/company/unisys> [image:
> Grey_TW] <http://twitter.com/unisyscorp> [image: Grey_YT]
> <http://www.youtube.com/theunisyschannel>[image: Grey_FB]
> <http://www.facebook.com/unisyscorp>[image: Grey_Vimeo]
> <https://vimeo.com/unisys>[image: Grey_UB] <http://blogs.unisys.com/>
>
>
>
>
>
>
>
>
