On 4/27/2012 8:33 PM, geeky2 wrote:
well, in this case when i say, "clean" (on the Master), i mean selecting the "Full Import with Cleaning" button from the DataImportHandler Development Console page in solr. at the top of the page, i have the check boxes selected for verbose and clean (*but i don't have the commit checkbox selected*).by doing the above process - doesn't this issue a deletion query - then start the import? and as a follow-up - when actually is the commit being done? here is my from my solrconfig.xml file on the master <updateHandler class="solr.DirectUpdateHandler2"> *<autoCommit> <maxTime>60000</maxTime> <maxDocs>1000</maxDocs> </autoCommit>* <maxPendingDeletes>100000</maxPendingDeletes> </updateHandler>
With commit turned off on the import, the *import* will not do a commit at any time, so something else has to do the commit or you will never see the new index.
In your case, you are relying on autocommit. Because I don't use autocommit, I can't say for sure that the following is right, but I believe that it is: With your settings during a full import, your index will go from having everything in it to having 1000 documents or less within one minute of the import starting.
If that is indeed what happens (and you should definitely test to make sure) and you have replication active, your slaves would have a reduced index that would slowly build back up as the import progressed on the master. I am pretty sure that's not what you want, so it is a good idea to disable replication until the full import is complete.
There is another option, one that would be a good idea if you make additions/deletions to your index on an interval that is smaller than the time it takes for a full-import: Maintain a live core and a build core on your master server. Build a new index in the build core while simultaneously keeping the live core up to date. When the build is complete, update it to be current and then swap the live core and build core. If replication is set up correctly, the slaves should replicate the new index as soon as the cores are swapped.
Thanks, Shawn
