Re: Solr CPU spiking up on bulk indexing

2019-06-18 Thread Erick Erickson
Dynamic fields don’t make any difference, they’re just like fixed fields as far as merging is concerned. So this is almost certainly merging being kicked off by your commits. The number of documents and the more terms, the more work Lucene has to do, so I suspect this is just how things work. I’l

Re: Solr CPU spiking up on bulk indexing

2019-06-18 Thread Venu
Thanks Erick. I see the above pattern only at the time of commit. I have many fields (like around 250 fields out of which around 100 fields are dynamic fields and around 3 n-gram fields and text fields, while many of them are stored fields along with indexed fields), will a merge take a lot of t

Re: Solr CPU spiking up on bulk indexing

2019-06-16 Thread Erick Erickson
When indexing, segments periodically merged by background threads, which can be quite CPU intensive. See: http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html Segment merges can be fairly long running, so even after indexing stops it can take some time for the CPU to s