On 2/24/2015 1:09 PM, Tang, Rebecca wrote: > Our solr index used to perform OK on our beta production box (anywhere > between 0-3 seconds to complete any query), but today I noticed that the > performance is very bad (queries take between 12 – 15 seconds). > > I haven't updated the solr index configuration (schema.xml/solrconfig.xml) > lately. All that's changed is the data — every month, I rebuild the solr > index from scratch and deploy it to the box. We will eventually go to > incremental builds. But for now, all indexes are built from scratch. > > Here are the stats: > Solr index size 183G > Documents in index 14364201 > We just have single solr box > It has 100G memory > 500G Harddrive > 16 cpus
The bottom line on this problem, and I'm sure it's not something you're going to want to hear: You don't have enough memory available to cache your index. I'd plan on at least 192GB of RAM for an index this size, and 256GB would be better. Depending on the exact index schema, the nature of your queries, and how large your Java heap for Solr is, 100GB of RAM could be enough for good performance on an index that size ... or it might be nowhere near enough. I would imagine that one of two things is true here, possibly both: 1) Your queries are very complex and involve accessing a very large percentage of the index data. 2) Your Java heap is enormous, leaving very little RAM for the OS to automatically cache the index. Adding more memory to the machine, if that's possible, might fix some of the problems. You can find a discussion of the problem here: http://wiki.apache.org/solr/SolrPerformanceProblems If you have any questions after reading that wiki article, feel free to ask them. Thanks, Shawn
