On Wed, 2016-01-13 at 15:01 -0700, Anria B. wrote: [256GB RAM]
> 1. Collection has 20-30 million docs. Just for completeness: How large is the collection in bytes? > 2. q=*&fq=someField:SomeVal ---> takes 2.5 seconds > 3. q=someField:SomeVal --> 300ms > 4. as numFound -> infinity, qtime -> infinity. What are you doing besides the q + fq above? Assuming a modest size index (let's say < 100GB), even 300ms for a simple key:value query is a long time. Usual culprits for performance problems when result set size grows are high values for rows, grouping and faceting. Could you experiment with q=queryA&fq=queryB vs q=queryA AND queryB to make sure that is is not the underlying queries themselves that causes the slowdown? Also, could you check the speed of the first request for q=queryA&fq=queryB vs subsequent requests (you might need to set the queryResultCache to zero to force a re-calculation of the result set), to see whether is is the creation of the fq result set or the intersection calculation that is slow? > We have already tested different autoCommit strategies, and different values > for heap size, starting at 16GB, 32GB, 64GB, 128GB ... The only place we > saw a 100ms improvement was between 32 - -Xmx=64GB. I would guess the 100 ms improvement was due to a factor not related to heap size. With the exception of a situation where the heap is nearly full, increasing Xmx will not improve Solr performance significantly. Quick note: Never set Xmx in the range 32GB-40GB (40GB is approximate): At the 32GB point, the JVM switches to larger pointers, which means that effective heap space is _smaller_ for Xmx=33GB than it is for Xmx=31GB: https://blog.codecentric.de/en/2014/02/35gb-heap-less-32gb-java-jvm-memory-oddities/ - Toke Eskildsen, State and University Library, Denmark
