Around 300MB will be spent for your filterCache and query cache (taking avg size for query string.. https://teaspoon-consulting.com/articles/solr-cache-tuning.html).
So during continous indexing and queries (complex) your cache thus heap utilization may go up. On Fri, Jul 7, 2017 at 9:41 AM, Markus Jelsma <markus.jel...@openindex.io> wrote: > Hello, > > Sorry, i forgot to mention we already run 6.6. > > Now i am looking at the sampler again on a freshly restarted instance. > This node has almost 6000 uncollectable TermQuery instances at this moment. > There are just 3.4 queries per second on this node. The final query is > complex but i cannot understand why so many TermQuery. Is this even normal? > I haven't seen this before, but i also have to say i never looked at this > specific thing before. > > Thanks, > Markus > > > -----Original message----- > > From:Erik Hatcher <erik.hatc...@gmail.com> > > Sent: Friday 7th July 2017 15:11 > > To: solr-user@lucene.apache.org > > Subject: Re: Slowly running OOM due to Query instances?! > > > > With generated Query’s, one has to be really careful with .equals and > .hashCode implementations. That may not be applicable here, but something > that has bitten me with caching. Note that there were fixes made in Solr > 6.6 with PayloadScoreQuery in this regard. See LUCENE-7808 and LUCENE-7481 > > > > Erik > > > > > > > On Jul 7, 2017, at 7:01 AM, Markus Jelsma <markus.jel...@openindex.io> > wrote: > > > > > > Hello, > > > > > > This morning i spotted our QTime suddenly go up. This has been going > on for a few hours by now and coincides with a serious increase in heap > consumption. No node ran out of memory so far but either that is going to > happen soon, or the nodes become unusable in another manner. > > > > > > I restarted one of the Solr instances and launched VisualVM at it, and > some other nodes that use to much heap. Starting the memory sampler, > something was obvious straight away. > > > > > > The nodes consuming too much heap all have a serious amount of *Query, > and BooleanClause instances, PayloadScoreQuery, TermQuery, BoostQuery, > BooleanQuery, SpanTermQuery and so forth. Lots of Builder and Term > instances too, very distinct from the node that was just freshly restarted. > > > > > > Another peculiarity, some nodes have exactly 65536 instances of > TermQuery and/or BoostQuery, probably unrelated but not something i would > have expected to see anyway. > > > > > > So, what's up? We do have a custom query parser extending > EdismaxQParser, it transliterates dates and creates payload and span > queries. I may be doing something wrong but i don't know, i have made and > used a variety of QParsers, for many years but this is new. Any hints on > where to look, what to watch out for? > > > > > > Many thanks! > > > Markus > > > > > > Xmx 800m, 8 GB RAM, SSD > > > 2 shards, three replica's > > > replica size ~17 GB, 2.2 million docs/replica > > > > >
