Fuad Efendi wrote: > Mark, > > > Nothing against orange-hat :) > > Nothing against GC tuning; but if SOLR needs application-specific settings > it should be well-documented. > > GC-tuning: for instance, we need it for 'realtime' Online Trading > applications. However, even Online Banking doesn't need; primary reason - GC > must happen 'outside of current transaction', GC 'must be predictable', and > (for instance) Oracle/BEA JRockit has specific 'realtime' version for > that... Does SOLR need that? > I'm not sure that Solr needs anything specific - but with a heap near 10 GB, you really do need some sort of parrallel or concurrent collection of the tenured space - unless you can live with the long pauses. I don't think thats Solr specific though.
> > Having load-stress simulator (multithreaded!!!) will definitely help to > predict any possible bottleneck... it's even better to write it from scratch > (depends on schema!), by sending random requests to SOLR in-parallel... > instead of waiting when FieldCache tries to add new FieldImpl to cache > (unpredictable!) > Yup - no argument from me here. Perhaps he does need more RAM and will find that out. Testing for that is a good idea. But by the sound of it, I just don't think we can guess that yet. I'm not against him testing to see though - its just semi a solution looking for a problem at the moment. It sounds like he is running this thing for hours and hours in a semi real environment (else why all the GC). He hasn't mentioned any need for more RAM yet. Again though, I'm not saying he shouldn't make sure he has enough RAM under any scenario. Everyone should. It just doesn't seem to be an issue hes indicated hes having. > > Tomcat is multithreaded; what if end-users need to load 1000s large > documents (in parallel! 1000s concurrent users), can you predict memory > requirements and GC options without application-specific knowledge? What > about new SOLR-Caches warming up? > > > -Fuad > > > >> -----Original Message----- >> From: Mark Miller [mailto:markrmil...@gmail.com] >> Sent: September-27-09 2:46 PM >> To: solr-user@lucene.apache.org >> Subject: Re: Solr and Garbage Collection >> >> If he needed double the RAM, he'd likely know by now :) The JVM likes to >> throw OOM exceptions when you need more RAM. Until it does - thats an >> odd path to focus on. There has been no indication he has ever seen an >> OOM with his over 10 GB heap. It sounds like he has run Solr in his >> environment for quite a long time - after running for that long, until >> he gets an OOM, its about as good as chasing ghost to worry about it. >> >> I like to think of GC tuning as orange-hat. Mostly because I like the >> color orange. >> >> Fuad Efendi wrote: >> >>>>> Ok. After the server ran for more than 12 hours, the time spent on GC >>>>> decreased from 11% to 3,4%, but 5 hours later it crashed. >>>>> >>>>> >>> All this 'black-hat' GC tuning and 'fast' object moving (especially >>> > objects > >>> accessing by some thread during GC-defragmentation) >>> >>> - try to use multithreaded load-stress tools (at least 100 requests >>> in-parallel) and see that you need at least double memory if 12Gb is >>> threshold for your FieldCache (largest objects) >>> >>> >>> Also, don't trust this counters: >>> >>> >>>> So I logged the Garbage Collection activity to check if it's because of >>>> that. It seems like 11% of the time the application runs, it is stopped >>>> because of GC. >>>> >>>> >>> Stopped? Of course, locking/unlocking in order to move objects currently >>> accessesd in multiuser-multithreaded Tomcat... you can easily create >>> > crash > >>> scenario proving that latest-greatest JVMs are buggy too. >>> >>> >>> >>> Don't forget: Tomcat is multithreaded, and if 'core' needs 10Gb in order >>> > to > >>> avoid OOM, you need to double it (in order to warm new cash instances on >>> index replica / update). >>> >>> >>> http://www.linkedin.com/in/liferay >>> >>> >>> >>> >> -- >> - Mark >> >> http://www.lucidimagination.com >> >> >> > > > > -- - Mark http://www.lucidimagination.com
