Hello, This morning i spotted our QTime suddenly go up. This has been going on for a few hours by now and coincides with a serious increase in heap consumption. No node ran out of memory so far but either that is going to happen soon, or the nodes become unusable in another manner.
I restarted one of the Solr instances and launched VisualVM at it, and some other nodes that use to much heap. Starting the memory sampler, something was obvious straight away. The nodes consuming too much heap all have a serious amount of *Query, and BooleanClause instances, PayloadScoreQuery, TermQuery, BoostQuery, BooleanQuery, SpanTermQuery and so forth. Lots of Builder and Term instances too, very distinct from the node that was just freshly restarted. Another peculiarity, some nodes have exactly 65536 instances of TermQuery and/or BoostQuery, probably unrelated but not something i would have expected to see anyway. So, what's up? We do have a custom query parser extending EdismaxQParser, it transliterates dates and creates payload and span queries. I may be doing something wrong but i don't know, i have made and used a variety of QParsers, for many years but this is new. Any hints on where to look, what to watch out for? Many thanks! Markus Xmx 800m, 8 GB RAM, SSD 2 shards, three replica's replica size ~17 GB, 2.2 million docs/replica
