We pass ExecutorService to Lucene's IndexSearcher at Amazon (for customer facing product search) and it's a big win on long-pole query latencies, but hurts red-line QPS (cluster capacity) a bit, due to less efficient collection across segments and thread context switching.
I'm surprised it's not an option for Solr and Elasticsearch ... for certain applications it's a huge win. And yes as David points out -- Collectors (CollectorManagers) need to support this "gather results for each segment separately then reduce in the end" mode... Mike McCandless http://blog.mikemccandless.com On Fri, Aug 30, 2019 at 4:45 PM David Smiley <david.w.smi...@gmail.com> wrote: > It'd take some work to do that. Years ago I recall Etsy did a POC and > shared their experience at Lucene/Solr Revolution in Washington DC; I > attended the presentation with great interest. One of the major obstacles, > if I recall, was the Collector needs to support this mode of operation, and > in particular Solr's means of flipping bits in a big bitset to accumulate > the DocSet had to be careful so that multiple threads don't try to > overwrite the same underlying "long" in the long[]. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley > > > On Mon, Aug 26, 2019 at 7:02 AM Aghasi Ghazaryan > <aghasi.ghazar...@picsart.com.invalid> wrote: > > > Hi, > > > > Lucene's IndexSearcher > > < > > > http://lucene.apache.org/core/8_0_0/core/org/apache/lucene/search/IndexSearcher.html#IndexSearcher-org.apache.lucene.index.IndexReaderContext-java.util.concurrent.ExecutorService- > > > > > supports > > running searches for each segment separately, using the provided > > ExecutorService. > > I wonder why SolrIndexSearcher does not support the same as it may > improve > > queries performance a lot? > > > > Thanks, looking forward to hearing from you. > > > > Regards > > Aghasi Ghazaryan > > >
