We have a very similar system. In our case we have a row version field in our sql database. When we run the full import we keep track of the latest row version at the time that the full import started. Once the full import is done we run an optimize and then run a delta import (actually this is a full import in a different DIH file, same mappings just with some different parameters to the stored procedure that we call). This catches the offline core up to live and then we swap.
Why can't you run the delta import on the rebuild core? Is there some specific issue that is preventing you? Cody -----Original Message----- From: KeesSchepers [mailto:k...@keesschepers.nl] Sent: Wednesday, March 14, 2012 11:59 AM To: solr-user@lucene.apache.org Subject: Solr core swap after rebuild in HA-setup / High-traffic Hello everybody, I am designing a new Solr architecture for one of my clients. This sorl architecture is for a high-traffic website with million of visitors but I am facing some design problems were I hope you guys could help me out. In my situation there are 4 Solr servers running, 1 server is master and 3 are slave. They are running Solr version 1.4. I use two cores 'live' and 'rebuild' and I use Solr DIH to rebuild a core which goes like this: 1. I wipe the reindex core 2. I run the DIH to the complete dataset (4 million documents) in peices of 20.000 records (to prevent very long mysql locks) 3. After the DIH is finished (2 hours) we have to also have to update the rebuild core with changes from the last two hours, this is a problem 4. After updating is done and the core is not more then some seconds behind we want to SWAP the cores. Everything goes well except for step 3. The rebuild and the core swap is all okay. Because the website is undergoing changes every minute we cannot pauze the delta-import on the live and walk behind for 2 hours. The problem is that I can't figure out a closing system with not delaying the live core to long and use the DIH instead of writing a lot of code. Did anyone face this problem before or could give me some tips? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-core-swap-after-rebuild-in-HA-setup-High-traffic-tp3826461p3826461.html Sent from the Solr - User mailing list archive at Nabble.com.
