Hi Stephen, regarding #1, can you verify following steps during backup/restore?
- Before backup command, make sure to run a "hard" commit on the original collection. The backup operation will capture only hard committed data. - After restore command, check the Solr web UI to verify that all replicas of the new (or restored) collection are in the "active" state. During my testing, I found that when one or more replicas are in "recovery" state, the doc count of the restored collection doesn't match the doc count of the original collection. But after the recovery is complete, the doc counts match. I will file a JIRA to fix this issue. Thanks Hrishikesh On Mon, Sep 26, 2016 at 9:34 AM, Stephen Weiss <steve.we...@wgsn.com> wrote: > #2 - that's great news. I'll try to patch it in and test it out. > > #1 - In all cases, the backup and restore both appear successful. There > are no failure messages for any of the shards, no warnings, etc - I didn't > even realize at first that data was missing until I noticed differences in > some of the query results when we were testing. Either manual restore of > the data or using the restore API (with all data on one node), we see the > same, so I think it's more a problem in the backup process than the restore > process. > > If there's any kind of debugging output we can provide that can help solve > this, let me know. > > -- > Steve > > On Sun, Sep 25, 2016 at 7:17 PM, Hrishikesh Gadre <gadre.s...@gmail.com> > wrote: > >> Hi Steve, >> >> Regarding the 2nd issue, a JIRA is already created and patch is uploaded >> (SOLR-9527). Can someone review and commit the patch? >> >> Regarding 1st issue, does backup command succeeds? Also do you see any >> warning/error log messages? How about the restore command? >> >> Thanks >> Hrishikesh >> >> >> >> On Sat, Sep 24, 2016 at 12:14 PM, Stephen Weiss <steve.we...@wgsn.com> >> wrote: >> >>> Hi everyone, >>> >>> We're very excited about SolrCloud's new backup / restore collection >>> APIs, which should introduce some major new efficiencies into our indexing >>> workflow. Unfortunately, we've run into some snags with it that are >>> preventing us from moving into production. I was hoping someone on the >>> list could help. >>> >>> 1) Data inconsistencies >>> >>> There seems to be a problem getting all the data consistently. >>> Sometimes, the backup will contain all of the data in the collection, and >>> sometimes, large portions of the collection (as much as 40%) will be >>> missing. We haven't quite figured out what might cause this yet, although >>> one thing I've noticed is the chances of success are greater when we are >>> only backing up one collection at a time. Unfortunately, for our workflow, >>> it will be difficult to make that work, and there still doesn't seem to be >>> a guarantee of success either way. >>> >>> 2) Shards are not distributed >>> >>> To make matters worse, for some reason, any type of restore operation >>> always seems to put all shards of the collection on the same node. We've >>> tried setting maxShardsPerNode to 1 in the restore command, but this has no >>> effect. We are seeing the same behavior on both 6.1 and 6.2.1. No matter >>> what we do, all the shards always go to the same node - and it's not even >>> the node that we execute the restore request on, but oddly enough, a >>> totally different node, and always the same one (the 4th one). It should >>> be noted that all nodes of our 8 node cloud are up and totally functional >>> when this happens. >>> >>> To work around this, we wrote up a quick script to create an empty >>> collection, which always distributes itself across the cloud quite well >>> (another indication that there's nothing wrong with the nodes themselves), >>> and then we rsync the individual shards' data into the empty shards and >>> reload the collection. This works fine, however, because of the data >>> inconsistencies mentioned above, we can't really move forward anyway. >>> >>> >>> Problem #2, we have a reasonable workaround for, but problem #1 we do >>> not. If anyone has any thoughts about either of these problems, I would be >>> very grateful. Thanks! >>> >>> -- >>> Steve >>> >>> ________________________________ >>> >>> WGSN is a global foresight business. Our experts provide deep insight >>> and analysis of consumer, fashion and design trends. We inspire our clients >>> to plan and trade their range with unparalleled confidence and accuracy. >>> Together, we Create Tomorrow. >>> >>> WGSN<http://www.wgsn.com/> is part of WGSN Limited, comprising of >>> market-leading products including WGSN.com<http://www.wgsn.com>, WGSN >>> Lifestyle & Interiors<http://www.wgsn.com/en/lifestyle-interiors>, WGSN >>> INstock<http://www.wgsninstock.com/>, WGSN StyleTrial< >>> http://www.wgsn.com/en/styletrial/> and WGSN Mindset< >>> http://www.wgsn.com/en/services/consultancy/>, our bespoke consultancy >>> services. >>> >>> The information in or attached to this email is confidential and may be >>> legally privileged. If you are not the intended recipient of this message, >>> any use, disclosure, copying, distribution or any action taken in reliance >>> on it is prohibited and may be unlawful. If you have received this message >>> in error, please notify the sender immediately by return email and delete >>> this message and any copies from your computer and network. WGSN does not >>> warrant that this email and any attachments are free from viruses and >>> accepts no liability for any loss resulting from infected email >>> transmissions. >>> >>> WGSN reserves the right to monitor all email through its networks. Any >>> views expressed may be those of the originator and not necessarily of WGSN. >>> WGSN is powered by Ascential plc<http://www.ascential.com>, which >>> transforms knowledge businesses to deliver exceptional performance. >>> >>> Please be advised all phone calls may be recorded for training and >>> quality purposes and by accepting and/or making calls from and/or to us you >>> acknowledge and agree to calls being recorded. >>> >>> WGSN Limited, Company number 4858491 >>> >>> registered address: >>> >>> Ascential plc, The Prow, 1 Wilder Walk, London W1B 5AP >>> >>> WGSN Inc., tax ID 04-3851246, registered office c/o National Registered >>> Agents, Inc., 160 Greentree Drive, Suite 101, Dover DE 19904, United States >>> >>> 4C Serviços de Informação Ltda., CNPJ/MF (Taxpayer's Register): >>> 15.536.968/0001-04, Address: Avenida Cidade Jardim, 377, 7˚ andar CEP >>> 01453-000, Itaim Bibi, São Paulo >>> >>> 4C Business Information Consulting (Shanghai) Co., Ltd, >>> 富新商务信息咨询（上海）有限公司, registered address Unit 4810/4811, 48/F Tower 1, Grand >>> Gateway, 1 Hong Qiao Road, Xuhui District, Shanghai >>> >> >> > > ------------------------------ > > WGSN is a global foresight business. Our experts provide deep insight and > analysis of consumer, fashion and design trends. We inspire our clients to > plan and trade their range with unparalleled confidence and accuracy. > Together, we Create Tomorrow. > > WGSN <http://www.wgsn.com/> is part of WGSN Limited, comprising of > market-leading products including WGSN.com <http://www.wgsn.com>, WGSN > Lifestyle & Interiors <http://www.wgsn.com/en/lifestyle-interiors>, WGSN > INstock <http://www.wgsninstock.com/>, WGSN StyleTrial > <http://www.wgsn.com/en/styletrial/> and WGSN Mindset > <http://www.wgsn.com/en/services/consultancy/>, our bespoke consultancy > services. > > The information in or attached to this email is confidential and may be > legally privileged. If you are not the intended recipient of this message, > any use, disclosure, copying, distribution or any action taken in reliance > on it is prohibited and may be unlawful. If you have received this message > in error, please notify the sender immediately by return email and delete > this message and any copies from your computer and network. WGSN does not > warrant that this email and any attachments are free from viruses and > accepts no liability for any loss resulting from infected email > transmissions. > > WGSN reserves the right to monitor all email through its networks. Any > views expressed may be those of the originator and not necessarily of WGSN. > WGSN is powered by Ascential plc <http://www.ascential.com>, which > transforms knowledge businesses to deliver exceptional performance. > > Please be advised all phone calls may be recorded for training and quality > purposes and by accepting and/or making calls from and/or to us you > acknowledge and agree to calls being recorded. > > WGSN Limited, Company number 4858491 > > registered address: > > Ascential plc, The Prow, 1 Wilder Walk, London W1B 5AP > > WGSN Inc., tax ID 04-3851246, registered office c/o National Registered > Agents, Inc., 160 Greentree Drive, Suite 101, Dover DE 19904, United States > > 4C Serviços de Informação Ltda., CNPJ/MF (Taxpayer's Register): > 15.536.968/0001-04, Address: Avenida Cidade Jardim, 377, 7˚ andar CEP > 01453-000, Itaim Bibi, São Paulo > > 4C Business Information Consulting (Shanghai) Co., Ltd, 富新商务信息咨询（上海）有限公司, > registered address Unit 4810/4811, 48/F Tower 1, Grand Gateway, 1 Hong Qiao > Road, Xuhui District, Shanghai >
