Your index is so small that it should easily get cached into OS memory as it is accessed. Having a too-big heap is a known problem situation.
https://cwiki.apache.org/confluence/display/SOLR/SolrPerformanceProblems#SolrPerformanceProblems-HowmuchheapspacedoIneed? On Tue, Oct 6, 2020 at 9:44 AM Karol Grzyb <grz...@gmail.com> wrote: > > Hi Matthew, > > Thank you for the answer, I cannot reproduce the setup locally I'll > try to convince them to reduce Xmx, I guess they will rather not agree > to 1GB but something less than 12G for sure. > And have some proper dev setup because for now we could only test prod > or stage which are difficult to adjust. > > Is being stuck in GC common behaviour when the index is small compared > to available heap during bigger load? I was more worried about the > ratio of heap to total host memory. > > Regards, > Karol > > > wt., 6 paź 2020 o 14:39 matthew sporleder <msporle...@gmail.com> napisał(a): > > > > You have a 12G heap for a 200MB index? Can you just try changing Xmx > > to, like, 1g ? > > > > On Tue, Oct 6, 2020 at 7:43 AM Karol Grzyb <grz...@gmail.com> wrote: > > > > > > Hi, > > > > > > I'm involved in investigation of issue that involves huge GC overhead > > > that happens during performance tests on Solr Nodes. Solr version is > > > 6.1. Last test were done on staging env, and we run into problems for > > > <100 requests/second. > > > > > > The size of the index itself is ~200MB ~ 50K docs > > > Index has small updates every 15min. > > > > > > > > > > > > Queries involve sorting and faceting. > > > > > > I've gathered some heap dumps, I can see from them that most of heap > > > memory is retained because of object of following classes: > > > > > > -org.apache.lucene.search.grouping.term.TermSecondPassGroupingCollector > > > (>4G, 91% of heap) > > > -org.apache.lucene.search.grouping.AbstractSecondPassGroupingCollector$SearchGroupDocs > > > -org.apache.lucene.search.FieldValueHitQueue$MultiComparatorsFieldValueHitQueue > > > -org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector > > > (>3.7G 76% of heap) > > > > > > > > > > > > Based on information above is there anything generic that can been > > > looked at as source of potential improvement without diving deeply > > > into schema and queries (which may be very difficlut to change at this > > > moment)? I don't see docvalues being enabled - could this help, as if > > > I get the docs correctly, it's specifically helpful when there are > > > many sorts/grouping/facets? Or I > > > > > > Additionaly I see, that many threads are blocked on LRUCache.get, > > > should I recomend switching to FastLRUCache? > > > > > > Also, I wonder if -Xmx12288m for java heap is not too much for 16G > > > memory? I see some (~5/s) page faults in Dynatrace during the biggest > > > traffic. > > > > > > Thank you very much for any help, > > > Kind regards, > > > Karol
