Hi, I dont run totally OOM (no OOM exceptions in the log) but I constantly garbage collect. While not collecting, SOLR master handels the updates pretty well.
Every insert is unique, so I dont have any deletes or optimizes and all queries are handled by the single slave instance. Is there a way to reduce the objects held in the old gen space ? It looks like the JVM is trying to hold as many objects as possible in the cache, to provide fast queries, who are not needed in my situation. Regarding the Jboss ... well as I said, its the minimalistic version of it and we use it due to automation process within our departement. In my test-env I tried it with a plain tomcat 6.x but without any improvements, so the Jboss overhead is minimal to nothing. The JVM parameters I wrote, are the ones I am struggling with at the moment. I was hoping someone will come up with a hint regarding the solarconfig.xml itself. PS: if anyone is questioning the implemented architecture (master -> slave, configs, schema, etc.) ... its our architects fault and I have to operate it ;-) 2010/10/15 Otis Gospodnetic <otis_gospodne...@yahoo.com> > Hello, > > I hope you are not running JBoss just to run Solr - there are simpler > containers > out there, e.g., Jetty. > Do you OOM? > Do things look better if you replicate less often (e.g. every 5 minutes > instead > of every 60 seconds)? > Do all/some of those -X__ JVM params actually help? > > Otis---- > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > ----- Original Message ---- > > From: accid <ac...@gmx.net> > > To: solr-user@lucene.apache.org > > Sent: Thu, October 14, 2010 1:25:34 PM > > Subject: Re: JVM GC troubles > > > > I forgot a few important details: > > > > solr version = 1.4.1 > > current index size = 50gb > > growth ~600mb / day > > jboss runs with web settings (same as minimal) > > 2010/10/14 <ac...@gmx.net> > > > > > Hi, > > > > > > as I am new here, I want to say hello and thanks in advance for your > help. > > > > > > > > > HW Setup: > > > > > > 1x SOLR Master - Sun Microsystems SUN FIRE X4450 - 4 x 2,93ghz, 64gb > ram > > > 1x SOLR Slave - Sun Microsystems SUN FIRE X4450 - 4 x 2,93ghz, 64gb > ram > > > > > > SW Setup: > > > > > > Solaris 10 Generic_142901-03 > > > jboss 5.1.0 > > > JDK 1.6 update 18 > > > > > > > > > # Specify the exact Java VM executable to use. > > > # > > > JAVA="/opt/appsrv/java6/bin/amd64/java" > > > > > > # > > > # Specify options to pass to the Java VM. > > > # > > > JAVA_OPTS="-server -Xms6144m -Xmx6144m -Xmn3072m > -XX:ThreadStackSize=1024 > > > -XX:MaxPermSize=512m -Dorg.jboss.resolver.warning=true > > > -Dsun.rmi.dgc.client.gcInterval=3600000 > > > -Dsun.rmi.dgc.server.gcInterval=3600000 > -Dnetworkaddress.cache.ttl=1800 > > > -XX:+UseConcMarkSweepGC" > > > > > > > > > SOLR Setup: > > > > > > #) the master has to deal an avg. update rate of 50 updates/s and > peaks of > > > 400 updates/s > > > > > > #) the slave replicates every 60s using the built in solr replication > > > method (NOT rsync) > > > > > > #) the slave querys are ~20/sec > > > > > > > > > #) schema.xml > > > > > > > > > <field name="myname1" type="string" indexed="true" stored="false" > > > required="true"/> > > > <field name="myname2" type="int" indexed="true" stored="true" > > > required="true"/> > > > <field name="myname3" type="int" indexed="true" stored="true" > > > required="true"/> > > > <field name="myname4" type="long" indexed="true" stored="true" > > > required="true"/> > > > <field name="myname5" type="int" indexed="true" stored="true" > > > required="true"/> > > > <field name="myname6" type="string" indexed="true" stored="true" > > > required="true"/> > > > <field name="myname7" type="string" indexed="true" stored="false"/> > > > <field name="myname8" type="string" indexed="true" stored="false"/> > > > <field name="myname9" type="string" indexed="true" stored="false"/> > > > <field name="myname10" type="long" indexed="true" stored="false"/> > > > <field name="myname11" type="int" indexed="true" stored="false"/> > > > <field name="myname12" type="string" indexed="true" stored="false"/> > > > <field name="myname13" type="tdate" indexed="true" stored="false"/> > > > <field name="myname14" type="int" indexed="true" stored="false" > > > multiValued="true"/> > > > <field name="myname15" type="string" indexed="true" stored="false" > > > multiValued="true"/> > > > <field name="myname16" type="int" indexed="true" stored="false" > > > multiValued="true"/> > > > <field name="myname17" type="string" indexed="true" stored="false" > > > multiValued="true"/> > > > <field name="myname18" type="string" indexed="true" stored="false" > > > multiValued="true"/> > > > <field name="myname19" type="string" indexed="true" stored="false" > > > multiValued="true"/> > > > <field name="myname20" type="boolean" indexed="true" stored="false"/> > > > <field name="myname21" type="int" indexed="true" stored="false" > > > required="true"/> > > > <field name="myname22" type="date" indexed="true" stored="true" > > > default="NOW" multiValued="false"/> > > > > > > > > > #) The solarconfig.xml is attached > > > > > > > > > > > > Both, master & slave suffer from serious performance impacts during > garbage > > > collects > > > > > > > > > I obviously have an GC problem, because ~30min after startup, the Old > space > > > is full and not beeing freed up. > > > > > > Below you find a JMX copy&paste of the Heap AFTER a garbage collect!! > As > > > you can see, even the Eden Space can only free up to 700mb total, > which > > > gives very little time to relax. The system does GC's 90% of the time. > > > > > > > > > > > > > > > Total Memory Pools: 5 > > > > > > Pool: Code Cache (Non-heap memory) > > > > > > Peak Usage : init:4194304, used:7679360, committed:7798784, > > > max:50331648 > > > Current Usage : init:4194304, used:7677312, committed:7798784, > > > max:50331648 > > > > > > > > > |---------| committed:7.44Mb > > > > > > > +---------------------------------------------------------------------+ > > > |/////////| | max:48Mb > > > > > > > +---------------------------------------------------------------------+ > > > |---------| used:7.32Mb > > > > > > > > > Pool: Par Eden Space (Heap memory) > > > > > > Peak Usage : init:2577006592, used:2577006592, > committed:2577006592, > > > max:2577006592 > > > Current Usage : init:2577006592, used:1911083120, > > > committed:2577006592, max:2577006592 > > > > > > > > > > > > > |---------------------------------------------------------------------| > > > committed:2.4Gb > > > > > > > +---------------------------------------------------------------------+ > > > |/////////////////////////////////////////////////// | > >max:2.4Gb > > > > > > > +---------------------------------------------------------------------+ > > > |--------------------------------------------------| > >used:1.78Gb > > > > > > > > > Pool: Par Survivor Space (Heap memory) > > > > > > Peak Usage : init:322109440, used:322109440, > committed:322109440, > > > max:322109440 > > > Current Usage : init:322109440, used:0, committed:322109440, > > > max:322109440 > > > > > > > > > > > > > |---------------------------------------------------------------------| > > > committed:307.19Mb > > > > > > > +---------------------------------------------------------------------+ > > > | | max:307.19Mb > > > > > > > +---------------------------------------------------------------------+ > > > | used:0b > > > > > > > > > Pool: CMS Old Gen (Heap memory) > > > > > > Peak Usage : init:3221225472, used:3221225472, > committed:3221225472, > > > max:3221225472 > > > Current Usage : init:3221225472, used:3207930400, > > > committed:3221225472, max:3221225472 > > > > > > > > > > > > > |---------------------------------------------------------------------| > > > committed:3Gb > > > > > > > +---------------------------------------------------------------------+ > > > > > > > |/////////////////////////////////////////////////////////////////////| > > > max:3Gb > > > > > > > +---------------------------------------------------------------------+ > > > > > > > |--------------------------------------------------------------------| > > > used:2.99Gb > > > > > > > > > Pool: CMS Perm Gen (Non-heap memory) > > > > > > Peak Usage : init:21757952, used:62622896, committed:104484864, > > > max:536870912 > > > Current Usage : init:21757952, used:62607888, > committed:104484864, > > > max:536870912 > > > > > > > > > |------------| committed:99.64Mb > > > > > > > +---------------------------------------------------------------------+ > > > |//////// | | max:512Mb > > > > > > > +---------------------------------------------------------------------+ > > > |-------| used:59.71Mb > > > > > > > > > > > > > > > I am looking forward for best practice values for both, solrconfig.xml > and > > > JVM settings. > > > > > > br > > > > > > accid > > > > > > -- > > > GRATIS! Movie-FLAT mit über 300 Videos. > > > Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome > > > > > >
