We decided to go with modern technology for the new cluster. CMS has been around for a very long time, maybe more then ten years.
These are the GC settings where we still use CMS. Instead of setting up a lot of ratios, I specify the sizes of the GC areas. That seems a lot more clear to me. We did some benchmarking and increasing the new space to 2G reduced the growth of tenured space. Most of Solr’s allocations have a lifetime of a single HTTP request. -Xms8g -Xmx8g -XX:NewSize=2g -XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent The last flag was because something was invoking a full GC to get accurate memory usage. That was annoying. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Dec 2, 2017, at 8:18 AM, Dominique Bejean <dominique.bej...@eolya.fr> > wrote: > > Hi Walter, > > Thank you for this response. Did you use CMS before G1 ? Was there any GC > issues fixed by G1 ? > > Dominique > > > Le sam. 2 déc. 2017 à 17:13, Walter Underwood <wun...@wunderwood.org> a > écrit : > >> We use an 8G heap and G1 with Shawn Heisey’s settings. Java 8, update 131. >> >> This has been solid in production with a 32 node Solr Cloud cluster. We do >> not do faceting. >> >> wunder >> Walter Underwood >> wun...@wunderwood.org >> http://observer.wunderwood.org/ (my blog) >> >> >>> On Dec 2, 2017, at 7:43 AM, Dominique Bejean <dominique.bej...@eolya.fr> >> wrote: >>> >>> Hi, >>> >>> I would like to have some advices on best practices related to Heap Size, >>> MMap, direct memory, GC algorithm and OS Swap. >>> >>> This is a waste subject and sorry for this long question but all these >>> items are linked in order to have a stable Solr environment. >>> >>> My understanding and questions. >>> >>> About JVM heap size setting >>> >>> JVM heap size setting is related to use case so there is no other advice >>> than reduce it at the minimum possible size in order to avoid GC issue. >>> Reduce Heap size at is minimum will be achieved mainly by : >>> >>> - >>> >>> Optimize schema by remove unused fields and not index / store fields if >>> it is not necessary >>> - >>> >>> Enable docValues on fields used for facetting, sorting and grouping >>> - >>> >>> Not oversize Solr cache >>> - >>> >>> Be careful with rows and fl query parameters >>> >>> >>> Any other advice is welcome :) >>> >>> >>> About MMap setting >>> >>> According to the great article “ >>> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html” >>> from Uwe Schindler, the only tasks that have to be done at OS settings >>> level is check that “ulimit -v” and “ulimit -m” both report “unlimited” >> and >>> increase vm.max_map_count setting from is default 65536. >>> >>> I suppose the best value is related to available off heap memory. I >>> generally set it to 262144. Is it a good value or is there a better way >> to >>> determine this value ? >>> >>> >>> About Direct Memory >>> >>> According to a response in Solr Maillig list from Uwe Schindler (again), >> I >>> understand that the MmapDirectory is not Direct Memory. >>> >>> The only place where I read that MaxDirectMemorySize JVM setting have to >> be >>> set for Solr is in Cloudera blog post and in Solr mailing list when using >>> Solr with HDFS. >>> >>> Is it necessary to change the default MaxDirectMemorySize JVM setting ? >> If >>> yes, how to determine the appropriate value ? >>> >>> >>> About OS Swap setting >>> >>> Linux generally starts swapping when less than 30% of the memory is free. >>> In order to avoid OS goes against Solr for off heap memory management, I >>> use to change OS swappiness value to 0. Can you confirm it is a good >> thing ? >>> >>> >>> About CMS GC vs G1 GC >>> >>> Default Solr setting use CMS GC. >>> >>> According to the post from Shawn Heisey in the old Solr wiki ( >>> https://wiki.apache.org/solr/ShawnHeisey), can we consider that G1 GC >> can >>> definitely be used with Solr for heap size over nearly 4Gb ? >>> >>> >>> Regards >>> >>> Dominique >>> >>> -- >>> Dominique Béjean >>> 06 08 46 12 43 >> >> -- > Dominique Béjean > 06 08 46 12 43
