Hello, I hope you are not running JBoss just to run Solr - there are simpler containers out there, e.g., Jetty. Do you OOM? Do things look better if you replicate less often (e.g. every 5 minutes instead of every 60 seconds)? Do all/some of those -X__ JVM params actually help?
Otis---- Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ ----- Original Message ---- > From: accid <ac...@gmx.net> > To: solr-user@lucene.apache.org > Sent: Thu, October 14, 2010 1:25:34 PM > Subject: Re: JVM GC troubles > > I forgot a few important details: > > solr version = 1.4.1 > current index size = 50gb > growth ~600mb / day > jboss runs with web settings (same as minimal) > 2010/10/14 <ac...@gmx.net> > > > Hi, > > > > as I am new here, I want to say hello and thanks in advance for your help. > > > > > > HW Setup: > > > > 1x SOLR Master - Sun Microsystems SUN FIRE X4450 - 4 x 2,93ghz, 64gb ram > > 1x SOLR Slave - Sun Microsystems SUN FIRE X4450 - 4 x 2,93ghz, 64gb ram > > > > SW Setup: > > > > Solaris 10 Generic_142901-03 > > jboss 5.1.0 > > JDK 1.6 update 18 > > > > > > # Specify the exact Java VM executable to use. > > # > > JAVA="/opt/appsrv/java6/bin/amd64/java" > > > > # > > # Specify options to pass to the Java VM. > > # > > JAVA_OPTS="-server -Xms6144m -Xmx6144m -Xmn3072m -XX:ThreadStackSize=1024 > > -XX:MaxPermSize=512m -Dorg.jboss.resolver.warning=true > > -Dsun.rmi.dgc.client.gcInterval=3600000 > > -Dsun.rmi.dgc.server.gcInterval=3600000 -Dnetworkaddress.cache.ttl=1800 > > -XX:+UseConcMarkSweepGC" > > > > > > SOLR Setup: > > > > #) the master has to deal an avg. update rate of 50 updates/s and peaks of > > 400 updates/s > > > > #) the slave replicates every 60s using the built in solr replication > > method (NOT rsync) > > > > #) the slave querys are ~20/sec > > > > > > #) schema.xml > > > > > > <field name="myname1" type="string" indexed="true" stored="false" > > required="true"/> > > <field name="myname2" type="int" indexed="true" stored="true" > > required="true"/> > > <field name="myname3" type="int" indexed="true" stored="true" > > required="true"/> > > <field name="myname4" type="long" indexed="true" stored="true" > > required="true"/> > > <field name="myname5" type="int" indexed="true" stored="true" > > required="true"/> > > <field name="myname6" type="string" indexed="true" stored="true" > > required="true"/> > > <field name="myname7" type="string" indexed="true" stored="false"/> > > <field name="myname8" type="string" indexed="true" stored="false"/> > > <field name="myname9" type="string" indexed="true" stored="false"/> > > <field name="myname10" type="long" indexed="true" stored="false"/> > > <field name="myname11" type="int" indexed="true" stored="false"/> > > <field name="myname12" type="string" indexed="true" stored="false"/> > > <field name="myname13" type="tdate" indexed="true" stored="false"/> > > <field name="myname14" type="int" indexed="true" stored="false" > > multiValued="true"/> > > <field name="myname15" type="string" indexed="true" stored="false" > > multiValued="true"/> > > <field name="myname16" type="int" indexed="true" stored="false" > > multiValued="true"/> > > <field name="myname17" type="string" indexed="true" stored="false" > > multiValued="true"/> > > <field name="myname18" type="string" indexed="true" stored="false" > > multiValued="true"/> > > <field name="myname19" type="string" indexed="true" stored="false" > > multiValued="true"/> > > <field name="myname20" type="boolean" indexed="true" stored="false"/> > > <field name="myname21" type="int" indexed="true" stored="false" > > required="true"/> > > <field name="myname22" type="date" indexed="true" stored="true" > > default="NOW" multiValued="false"/> > > > > > > #) The solarconfig.xml is attached > > > > > > > > Both, master & slave suffer from serious performance impacts during garbage > > collects > > > > > > I obviously have an GC problem, because ~30min after startup, the Old space > > is full and not beeing freed up. > > > > Below you find a JMX copy&paste of the Heap AFTER a garbage collect!! As > > you can see, even the Eden Space can only free up to 700mb total, which > > gives very little time to relax. The system does GC's 90% of the time. > > > > > > > > > > Total Memory Pools: 5 > > > > Pool: Code Cache (Non-heap memory) > > > > Peak Usage : init:4194304, used:7679360, committed:7798784, > > max:50331648 > > Current Usage : init:4194304, used:7677312, committed:7798784, > > max:50331648 > > > > > > |---------| committed:7.44Mb > > > > +---------------------------------------------------------------------+ > > |/////////| | max:48Mb > > > > +---------------------------------------------------------------------+ > > |---------| used:7.32Mb > > > > > > Pool: Par Eden Space (Heap memory) > > > > Peak Usage : init:2577006592, used:2577006592, committed:2577006592, > > max:2577006592 > > Current Usage : init:2577006592, used:1911083120, > > committed:2577006592, max:2577006592 > > > > > > > > |---------------------------------------------------------------------| > > committed:2.4Gb > > > > +---------------------------------------------------------------------+ > > |/////////////////////////////////////////////////// | >max:2.4Gb > > > > +---------------------------------------------------------------------+ > > |--------------------------------------------------| >used:1.78Gb > > > > > > Pool: Par Survivor Space (Heap memory) > > > > Peak Usage : init:322109440, used:322109440, committed:322109440, > > max:322109440 > > Current Usage : init:322109440, used:0, committed:322109440, > > max:322109440 > > > > > > > > |---------------------------------------------------------------------| > > committed:307.19Mb > > > > +---------------------------------------------------------------------+ > > | | max:307.19Mb > > > > +---------------------------------------------------------------------+ > > | used:0b > > > > > > Pool: CMS Old Gen (Heap memory) > > > > Peak Usage : init:3221225472, used:3221225472, committed:3221225472, > > max:3221225472 > > Current Usage : init:3221225472, used:3207930400, > > committed:3221225472, max:3221225472 > > > > > > > > |---------------------------------------------------------------------| > > committed:3Gb > > > > +---------------------------------------------------------------------+ > > > > |/////////////////////////////////////////////////////////////////////| > > max:3Gb > > > > +---------------------------------------------------------------------+ > > > > |--------------------------------------------------------------------| > > used:2.99Gb > > > > > > Pool: CMS Perm Gen (Non-heap memory) > > > > Peak Usage : init:21757952, used:62622896, committed:104484864, > > max:536870912 > > Current Usage : init:21757952, used:62607888, committed:104484864, > > max:536870912 > > > > > > |------------| committed:99.64Mb > > > > +---------------------------------------------------------------------+ > > |//////// | | max:512Mb > > > > +---------------------------------------------------------------------+ > > |-------| used:59.71Mb > > > > > > > > > > I am looking forward for best practice values for both, solrconfig.xml and > > JVM settings. > > > > br > > > > accid > > > > -- > > GRATIS! Movie-FLAT mit über 300 Videos. > > Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome > > >
