Mark,
Nothing against orange-hat :) Nothing against GC tuning; but if SOLR needs application-specific settings it should be well-documented. GC-tuning: for instance, we need it for 'realtime' Online Trading applications. However, even Online Banking doesn't need; primary reason - GC must happen 'outside of current transaction', GC 'must be predictable', and (for instance) Oracle/BEA JRockit has specific 'realtime' version for that... Does SOLR need that? Having load-stress simulator (multithreaded!!!) will definitely help to predict any possible bottleneck... it's even better to write it from scratch (depends on schema!), by sending random requests to SOLR in-parallel... instead of waiting when FieldCache tries to add new FieldImpl to cache (unpredictable!) Tomcat is multithreaded; what if end-users need to load 1000s large documents (in parallel! 1000s concurrent users), can you predict memory requirements and GC options without application-specific knowledge? What about new SOLR-Caches warming up? -Fuad > -----Original Message----- > From: Mark Miller [mailto:markrmil...@gmail.com] > Sent: September-27-09 2:46 PM > To: solr-user@lucene.apache.org > Subject: Re: Solr and Garbage Collection > > If he needed double the RAM, he'd likely know by now :) The JVM likes to > throw OOM exceptions when you need more RAM. Until it does - thats an > odd path to focus on. There has been no indication he has ever seen an > OOM with his over 10 GB heap. It sounds like he has run Solr in his > environment for quite a long time - after running for that long, until > he gets an OOM, its about as good as chasing ghost to worry about it. > > I like to think of GC tuning as orange-hat. Mostly because I like the > color orange. > > Fuad Efendi wrote: > >>> Ok. After the server ran for more than 12 hours, the time spent on GC > >>> decreased from 11% to 3,4%, but 5 hours later it crashed. > >>> > > > > All this 'black-hat' GC tuning and 'fast' object moving (especially objects > > accessing by some thread during GC-defragmentation) > > > > - try to use multithreaded load-stress tools (at least 100 requests > > in-parallel) and see that you need at least double memory if 12Gb is > > threshold for your FieldCache (largest objects) > > > > > > Also, don't trust this counters: > > > >> So I logged the Garbage Collection activity to check if it's because of > >> that. It seems like 11% of the time the application runs, it is stopped > >> because of GC. > >> > > > > > > Stopped? Of course, locking/unlocking in order to move objects currently > > accessesd in multiuser-multithreaded Tomcat... you can easily create crash > > scenario proving that latest-greatest JVMs are buggy too. > > > > > > > > Don't forget: Tomcat is multithreaded, and if 'core' needs 10Gb in order to > > avoid OOM, you need to double it (in order to warm new cash instances on > > index replica / update). > > > > > > http://www.linkedin.com/in/liferay > > > > > > > > > -- > - Mark > > http://www.lucidimagination.com > >
