If you are not using NRT readers then the applyAllDeletes/writeAllDeletes boolean values are completely unused (and should have no impact on your performance).
Mike McCandless http://blog.mikemccandless.com On Sun, May 28, 2017 at 8:34 PM, Nawab Zada Asad Iqbal <khi...@gmail.com> wrote: > After reading some more code it seems if we are sure that there are no > deletes in this segment/index, then setting applyAllDeletes and > writeAllDeletes both to false will achieve similar to what I was getting in > 4.5.0 > > However, after I read the comment from IndexWriter::DirectoryReader > getReader(boolean applyAllDeletes, boolean writeAllDeletes) , it seems that > this method is particular to NRT. Since we are not using soft commits, can > this change actually improve our performance during full reindex? > > > Thanks > Nawab > > > > > > > > > > On Sun, May 28, 2017 at 2:16 PM, Nawab Zada Asad Iqbal <khi...@gmail.com> > wrote: > >> Thanks Michael and Shawn for the detailed response. I was later able to >> pull the full history using gitk; and found the commits behind this patch. >> >> Mike: >> >> So, in solr 4.5.0 ; some earlier developer has added code and config to >> set applyAllDeletes to false when we reindex all the data. At the moment, >> I am not sure about the performance gain by this. >> >> <!-- Whether to check and apply deletes on flushing of segments. >> Defaults to true and it only makes sense to set this to false >> if you know for sure that there are no deletes performed on the >> shard, i.e. when performing a full index --> >> >> >> I am investigating the question, if this change is still needed in 6.5.1 >> or can this be achieved by any other configuration? >> >> For now, we are not planning to use NRT and solrCloud. >> >> >> Thanks >> Nawab >> >> On Sun, May 28, 2017 at 9:26 AM, Michael McCandless < >> luc...@mikemccandless.com> wrote: >> >>> Sorry, yes, that commit was one of many on a feature branch I used to >>> work on LUCENE-5438, which added near-real-time index replication to >>> Lucene. Before this change, Lucene's replication module required a commit >>> in order to replicate, which is a heavy operation. >>> >>> The writeAllDeletes boolean option asks Lucene to move all recent >>> deletes (tombstone bitsets) to disk while opening the NRT (near-real-time) >>> reader. >>> >>> Normally Lucene won't always do that, and will instead carry the bitsets >>> in memory from writer to reader, for reduced refresh latency. >>> >>> What sort of custom changes do you have in this part of Lucene? >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> On Sat, May 27, 2017 at 10:35 PM, Nawab Zada Asad Iqbal < >>> khi...@gmail.com> wrote: >>> >>>> Hi all >>>> >>>> I am looking at following change in lucene-solr which doen't mention any >>>> JIRA. How can I know more about it? >>>> >>>> "1ae7291 Mike McCandless on 1/24/16 at 3:17 PM current patch" >>>> >>>> Specifically, I am interested in what 'writeAllDeletes' does in the >>>> following method. Let me know if it is very stupid question and I should >>>> have done something else before emailing here. >>>> >>>> static DirectoryReader open(IndexWriter writer, SegmentInfos infos, >>>> boolean applyAllDeletes, boolean writeAllDeletes) throws IOException { >>>> >>>> Background: We are running solr4.5 and upgrading to 6.5.1. We have >>>> some custom code in this area, which we need to merge. >>>> >>>> >>>> Thanks >>>> >>>> Nawab >>>> >>> >>> >> >
