On 9/22/2015 11:54 AM, vsilgalis wrote: > I've actually read that article a few times. > > Yeah I know we aren't perfect in opening searchers. Yes we are committing > from the client, this is something that is changing in our next code > release, AND we are auto soft committing every second. > > <filterCache class="solr.FastLRUCache" size="32768" initialSize="32768" > autowarmCount="256"/> > <queryResultCache class="solr.LRUCache" size="32768" initialSize="32768" > autowarmCount="256"/> > <documentCache class="solr.LRUCache" size="32768" initialSize="32768" > autowarmCount="256"/>
Those are huge caches. Especially the filterCache, because each filter entry can be megabytes in size, depending on how many documents are in the core. If your index ever reaches the point where the filterCache can grow to thousands of entries, your heap memory usage may grow out of control. The documentCache cannot autowarm, so that autowarmCount setting is irrelevant. The other two are important, and 256 is a pretty large number for that setting. It is unlikely that your autowarming completes in less than one second. I've repeated some of what Erick already told you, but I would like to add the following. On your autoSoftCommit interval of one second, the article that Erick linked has this to say: ------- Set your soft commit interval to as long as you can stand. Don't listen to your product manager who says "we need no more than 1 second latency". Really. Push back hard and see if the /user/ is best served or will even notice. Soft commits and NRT are pretty amazing, but they’re not free. ------- This autoSoftCommit interval, especially with large indexes, can cause a performance death spiral. In SolrCloud, that death spiral tends to cause constant replica recovery. A previous message you sent to the list indicated that your shards are each 10GB in size, which counts as a large index. Many people have indexes that are larger, but that's still pretty big. Thanks, Shawn
