Hello, There has not been a change in code recently. An increase in QPS and a broader spectrum of queries (a world wide network of users started using our platform) happened a month ago. All still happy with 400 MB, nothing changed. The index does increase continously, all the time so i did expect more BytesRef and stuff to be around, not Query instances, that's just weird.
Thanks, Markus -----Original message----- > From:Joel Bernstein <joels...@gmail.com> > Sent: Friday 7th July 2017 14:50 > To: solr-user@lucene.apache.org > Subject: Re: Slowly running OOM due to Query instances?! > > What changed in the system? > > Has there been a code change, increased QPS or different types of queries > being run? > > > > Joel Bernstein > http://joelsolr.blogspot.com/ > > On Fri, Jul 7, 2017 at 8:07 AM, Markus Jelsma <markus.jel...@openindex.io> > wrote: > > > Hello, > > > > No, 800 MB heap is fine. Up till now, heap consumption averages at 400 MB > > for past few months, which is fine. FilterCache and QueryCache are always > > full. If restarted, they fill up very quickly. FilterCache is just 512 > > entries large, QueryCache about 2k. > > > > All Solr instances run on separate machines (VMs). > > > > Thanks, > > Markus > > > > > > > > -----Original message----- > > > From:Susheel Kumar <susheel2...@gmail.com> > > > Sent: Friday 7th July 2017 13:27 > > > To: solr-user@lucene.apache.org > > > Subject: Re: Slowly running OOM due to Query instances?! > > > > > > Xms 800m sounds low regardless did you know how much total cache > > > consumption may go based on your current solrconfig.xml settings. Also 2 > > > shards and 3 replca's are on 6 such machines i assume. > > > > > > Thanks, > > > Susheel > > > > > > On Fri, Jul 7, 2017 at 7:01 AM, Markus Jelsma < > > markus.jel...@openindex.io> > > > wrote: > > > > > > > Hello, > > > > > > > > This morning i spotted our QTime suddenly go up. This has been going on > > > > for a few hours by now and coincides with a serious increase in heap > > > > consumption. No node ran out of memory so far but either that is going > > to > > > > happen soon, or the nodes become unusable in another manner. > > > > > > > > I restarted one of the Solr instances and launched VisualVM at it, and > > > > some other nodes that use to much heap. Starting the memory sampler, > > > > something was obvious straight away. > > > > > > > > The nodes consuming too much heap all have a serious amount of *Query, > > and > > > > BooleanClause instances, PayloadScoreQuery, TermQuery, BoostQuery, > > > > BooleanQuery, SpanTermQuery and so forth. Lots of Builder and Term > > > > instances too, very distinct from the node that was just freshly > > restarted. > > > > > > > > Another peculiarity, some nodes have exactly 65536 instances of > > TermQuery > > > > and/or BoostQuery, probably unrelated but not something i would have > > > > expected to see anyway. > > > > > > > > So, what's up? We do have a custom query parser extending > > EdismaxQParser, > > > > it transliterates dates and creates payload and span queries. I may be > > > > doing something wrong but i don't know, i have made and used a variety > > of > > > > QParsers, for many years but this is new. Any hints on where to look, > > what > > > > to watch out for? > > > > > > > > Many thanks! > > > > Markus > > > > > > > > Xmx 800m, 8 GB RAM, SSD > > > > 2 shards, three replica's > > > > replica size ~17 GB, 2.2 million docs/replica > > > > > > > > > >
