Re: Solr7: Bad query throughput around commit time

2017-11-12 Thread Nawab Zada Asad Iqbal
Thanks Erik, Yes i see the 'sawtooth' pattern. I will try your suggestion, but i am wondering why were the queries performant with solr4 without DocValues? Have some defaults changed? --- On Sat, Nov 11, 2017 at 8:28 PM, Erick Erickson wrote: > Nawab: > > bq: Cache hit ratios are all in 80+%

Re: Solr7: Bad query throughput around commit time

2017-11-11 Thread Erick Erickson
Nawab: bq: Cache hit ratios are all in 80+% (even when i decreased the filterCache to 128) This suggests that you use a relatively small handful of fq clauses, which is perfectly fine. Having 450M docs and a cache size of 1024 is _really_ scary! You had a potential for a 57G (yes, gigabyte) filte

Re: Solr7: Bad query throughput around commit time

2017-11-11 Thread Nawab Zada Asad Iqbal
~248 gb Nawab On Sat, Nov 11, 2017 at 2:41 PM Kevin Risden wrote: > > One machine runs with a 3TB drive, running 3 solr processes (each with > one core as described above). > > How much total memory on the machine? > > Kevin Risden > > On Sat, Nov 11, 2017 at 1:08 PM, Nawab Zada Asad Iqbal >

Re: Solr7: Bad query throughput around commit time

2017-11-11 Thread Kevin Risden
> One machine runs with a 3TB drive, running 3 solr processes (each with one core as described above). How much total memory on the machine? Kevin Risden On Sat, Nov 11, 2017 at 1:08 PM, Nawab Zada Asad Iqbal wrote: > Thanks for a quick and detailed response, Erick! > > Unfortunately i don't h

Re: Solr7: Bad query throughput around commit time

2017-11-11 Thread Nawab Zada Asad Iqbal
Thanks for a quick and detailed response, Erick! Unfortunately i don't have a proof, but our servers with solr 4.5 are running really nicely with the above config. I had assumed that same or similar settings will also perform well with Solr 7, but that assumption didn't hold. As, a lot has change

Re: Solr7: Bad query throughput around commit time

2017-11-09 Thread Erick Erickson
What evidence to you have that the changes you've made to your configs are useful? There's lots of things in here that are suspect: 1 First, this is useless unless you are forceMerging/optimizing. Which you shouldn't be doing under most circumstances. And you're going to be rewriting a lot of d

Solr7: Bad query throughput around commit time

2017-11-09 Thread Nawab Zada Asad Iqbal
Hi, I am committing every 5 minutes using a periodic cron job "curl http://localhost:8984/solr/core1/update?commit=true";. Besides this, my app doesn't do any soft or hard commits. With Solr 7 upgrade, I am noticing that query throughput plummets every 5 minutes - probably when the commit happens