Well, the reason i want to control number of indexing threads is to restrict number of "segments" being created at one time in the RAM. One indexing thread in lucene corresponds to one segment being written. I need a fine control on the number of segments. Less than that, and I will not be fully utilizing my writing capacity. On the other hand, if I have more threads, then I will end up a lot more segments of small size, which I will need to flush frequently and then merge, and that will cause a different kind of problem.
Your suggestion will require me and other such solr users to create a tight coupling between the clients and the Solr servers. My client is not SolrJ based. IN a scenario when I am connecting and indexing to Solr remotely, I want more requests to be waiting on the solr side so that they start writing as soon as an Indexing thread is available, vs waiting on my client side - on the other side of the wire. Thanks Nawab On Wed, Nov 1, 2017 at 7:11 AM, Shawn Heisey <apa...@elyograg.org> wrote: > On 10/31/2017 4:57 PM, Nawab Zada Asad Iqbal wrote: > >> I hit this issue https://issues.apache.org/jira/browse/SOLR-11504 while >> migrating to solr6 and locally working around it in Lucene code. I am >> thinking to fix it properly and hopefully patch back to Solr. Since, >> Lucene >> code does not want to keep any such config, I am thinking to use a >> counting >> semaphore in Solr code before calling IndexWriter.addDocument(s) or >> IndexWriter.updateDocument(s). >> > > There's a fairly simple way to control the number of indexing threads that > doesn't require ANY changes to Solr: Don't start as many threads/processes > on your indexing client(s). If you control the number of simultaneous > requests sent to Solr, then Solr won't start as many indexing threads. > That kind of control over your indexing system is something that's always > preferable to have. > > Thanks, > Shawn >
