Hi Jean-Sebastien, One thing you didn't mention is whether as you are increasing(I assume) cache sizes you actually see performance improve? If not, then maybe there is no value increasing cache sizes.
I assume you changed only one cache at a time? Were you able to get any one of them to the point where there were no evictions without things breaking? What are your queries like, can you share a few examples? Otis -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Mon, Jun 2, 2014 at 11:09 AM, Jean-Sebastien Vachon < jean-sebastien.vac...@wantedanalytics.com> wrote: > Thanks for your quick response. > > Our JVM is configured with a heap of 8GB. So we are pretty close of the > "optimal" configuration you are mentioning. The only other programs running > is Zookeeper (which has its own storage device) and a proprietary API (with > a heap of 1GB) we have on top of Solr to server our customer`s requests. > > I will look into the filterCache to see if we can better use it. > > Thanks for your help > > > -----Original Message----- > > From: Shawn Heisey [mailto:s...@elyograg.org] > > Sent: June-02-14 10:48 AM > > To: solr-user@lucene.apache.org > > Subject: Re: Strange behaviour when tuning the caches > > > > On 6/2/2014 8:24 AM, Jean-Sebastien Vachon wrote: > > > We have yet to determine where the exact breaking point is. > > > > > > The two patterns we are seeing are: > > > > > > - less cache (around 20-30% hit/ratio), poor performance but > > > overall good stability > > > > When caches are too small, a low hit ratio is expected. Increasing them > is a > > good idea, but only increase them a little bit at a time. The > filterCache in > > particular should not be increased dramatically, especially the > > autowarmCount value. Filters can take a very long time to execute, so a > high > > autowarmCount can result in commits taking forever. > > > > Each filter entry can take up a lot of heap memory -- in terms of bytes, > it is > > the number of documents in the core divided by 8. This means that if the > > core has 10 million documents, each filter entry (for JUST that > > core) will take over a megabyte of RAM. > > > > > - more cache (over 90% hit/ratio), improved performance but > > > almost no stability. In that case, we start seeing messages such as > > > "No shards hosting shard X" or "cancelElection did not find election > > > node to remove" > > > > This would not be a direct result of increasing the cache size, unless > perhaps > > you've increased them so they are *REALLY* big and you're running out of > > RAM for the heap or OS disk cache. > > > > > Anyone, has any advice on what could cause this? I am beginning to > > > suspect the JVM version, is there any minimal requirements regarding > > > the JVM? > > > > Oracle Java 7 is recommended for all releases, and required for Solr > 4.8. You > > just need to stay away from 7u40, 7u45, and 7u51 because of bugs in Java > > itself. Right now, the latest release is recommended, which is 7u60. > The > > 7u21 release that you are running should be perfectly fine. > > > > With six 9.4GB cores per node, you'll achieve the best performance if you > > have about 60GB of RAM left over for the OS disk cache to use -- the > size of > > your index data on disk. You did mention that you have 92GB of RAM per > > node, but you have not said how big your Java heap is, or whether there > is > > other software on the machine that may be eating up RAM for its heap or > > data. > > > > http://wiki.apache.org/solr/SolrPerformanceProblems > > > > Thanks, > > Shawn > > > > ----- > > Aucun virus trouvé dans ce message. > > Analyse effectuée par AVG - www.avg.fr > > Version: 2014.0.4570 / Base de données virale: 3950/7571 - Date: > > 27/05/2014 >
