On 8/14/2013 9:09 AM, Jean-Sebastien Vachon wrote:
I am running some benchmarks to tune our Solr 4.3 cloud and noticed that while the reported QTime is quite satisfactory (100 ms or so), the elapsed time is quite large (around 5 seconds). The collection contains 12.8M documents and the index size on disk is about 35 GB.. I have only one shard and 4 replicas (we intent to have 5 shards but wanted to see how Solr would perform with only one shard so that we could benefit from all Solr functions)
As your other reply from Scott says, you may be dealing with the fact that Solr must fetch stored field data from the index on disk and decompress it. Solr 4.1 and later have compressed stored fields. There is no way other than writing custom Solr code to turn off the compression. If the documents are very large, the decompression step can be a big performance penalty.
You have a VERY large field list - fl parameter. Have you tried just leaving that parameter off so that Solr will return all stored fields instead of identifying each field? This might not help at all, I'm just putting it out there as something to try.
You also have grouping enabled. From what I understand, that can be slow. If you turn that off, what happens to your elapsed times?
Your free RAM vs. index size is good, assuming that there's nothing else on your Solr servers. With 12.8 million documents plus the use of grouping and sorting, you might need a larger java heap. Try increasing it to 5GB as an initial test and see if that makes any difference, either good or bad.
Your email says you checked for huge GC, but without knowing exactly how you checked, it's difficult to know what you would have actually found.
Thanks, Shawn
