I believe I figured out what the issue is. Even though we do not open a new searcher on master during full import, the slave anyway replicates the index after auto commits! (Is this desired behavior?) Since "clean=true" this meant all the docs were deleted on slave and a partial index got replicated! The reason only the date query did not return any results is because recently created docs have higher doc IDs and we index by ascending order of IDs!
I believe I have two options: - as Chris suggested I have to use "clean=false" so the existing docs are not deleted first on the slave. Since we have primary keys, newly added docs will overwrite old docs as they get added. - disable replication after commits. Replicate only after optimize. Thx all for your help. On Fri, Feb 28, 2014 at 8:06 PM, Arun Rangarajan <arunrangara...@gmail.com>wrote: > Thx, Erick and Chris. > > This is indeed very strange. Other queries which do not restrict by the > date field are returning results, so the index is definitely not empty. Has > it got something to do with the date query part, with NOW/DAY or something > in here? > first_publish_date:[NOW/DAY-33DAYS TO NOW/DAY-3DAYS] > > For now, I have set up a script to just log the number of docs on the > slave every minute. Will monitor and report the findings. > > > On Fri, Feb 28, 2014 at 6:49 PM, Chris Hostetter <hossman_luc...@fucit.org > > wrote: > >> >> : This is odd. The full import, I think, deletes the >> : docs in the index when it starts. >> >> Yeah, if you are doing a full-import everyday, and you don't want it to >> delete all docs when it starts, you need to specify "clearn=false" >> >> >> https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler#UploadingStructuredDataStoreDatawiththeDataImportHandler-Parametersforthefull-importCommand >> >> >> >> -Hoss >> http://www.lucidworks.com/ >> > >
