Hi there,

recently I had few Java OOM in my Solr 4.8.1 instance.

Here the configuration I have.

-Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
-Dsolr.log=/opt/tomcat/logs
-DzkHost=ep-1:2181,ep-2:2181,ep-3:2181
-Dsolr.solr.home=/store/solr
-Xms2g -Xmx16g
-server
-XX:+UseG1GC
-XX:+ParallelRefProcEnabled
-XX:G1HeapRegionSize=8m
-XX:MaxGCPauseMillis=400
-XX:+UseLargePages
-XX:+AggressiveOpts
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/opt/tomcat/dumpoom/dump.hprof
-Dcom.sun.management.jmxremote.port=1616
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.rmi.port=1616
-Dcom.sun.management.jmxremote.local.only=false
-Djava.rmi.server.hostname=localhost
-Djava.endorsed.dirs=/opt/tomcat/endorsed

This is the error:

org.apache.solr.common.SolrException: Error while processing facet fields:
java.lang.OutOfMemoryError: Java heap space

Here the complete stacktrace:
https://gist.github.com/freedev/a14aa9e6ae33fc3ddb2f02d602b34e2b

I suppose these errors are generated by an increase of traffic coming from
crawlers/spiders.
So given the sudden appear of these errors, I've configured a memory dump
of jvm in case of oom.

Analyzing the memory dump with the Eclipse Memory Analizer and running the
Usual argh... :) the "Leak Suspects Report" I've found that about 80% of
memory was occupied by one instance of FieldCacheImpl :

One instance of "org.apache.lucene.search.FieldCacheImpl" loaded by
"org.apache.catalina.loader.WebappClassLoader @ 0x3c145b028" occupies
8,248,329,008 (79.69%) bytes. The memory is accumulated in one instance of
"java.util.WeakHashMap$Entry[]" loaded by "<system class loader>".

I was unable to understand what field was, it seems to be a float.

Anyone has an advice to give me? For long time this server has worked well,
without problems. Recently we have a huge traffic coming from
spiders/crawlers but I don't understand how these requests can consume all
the available memory.

Best regards,
Vincenzo

-- 
Vincenzo D'Amore

Reply via email to