1. yes, I'm sure that pauses are due to GCs. I monitor the cluster and receive continuously metric from system and from java process. I see clearly that when soft commit is triggered, major GCs start occurring (sometimes reocuuring on the same process) and latency rises. I use CMS GC and jdk 1.7.75
2. My previous post was about another use case, but nevertheless I have configured docvalues in the faceted fields. Toke Eskildsen wrote > On Wed, 2015-05-06 at 00:58 -0700, adfel70 wrote: >> each shard has around 200 million docs. size of each shard is 250GB. >> this runs on 12 machines. each machine has 4 SSD disks and 4 solr >> processes. >> each process has 28GB heap. each machine has 196GB RAM. > > [...] > >> 1. heavy GCs when soft commit is performed (methods 1,2) or when >> hardcommit >> opensearcher=true is performed. these GCs cause heavy latency (average >> latency is 3 secs. latency during the problem is 80secs) > > Sanity check: Are you sure the pauses are due to garbage collection? > > You have a fairly large heap and judging from your previous post > "problem with facets - out of memory exception", you are doing > non-trivial faceting. Are you using DocValues, as Marc suggested? > > > - Toke Eskildsen, State and University Library, Denmark -- View this message in context: http://lucene.472066.n3.nabble.com/severe-problems-with-soft-and-hard-commits-in-a-large-index-tp4204068p4204088.html Sent from the Solr - User mailing list archive at Nabble.com.
