Solr :: Snapshooter Cannot allocate memory

[OLX - Pablo Garrido]

Hello

        We are facing this recurrent error on our Master Solr Server
every now and then :

SEVERE: java.io.IOException: Cannot run program
"/opt/solr/bin/snapshooter" (in directory "solr/bin"):
java.io.IOException: error=12, Cannot allocate memory
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
        at java.lang.Runtime.exec(Runtime.java:593)
        at
org.apache.solr.core.RunExecutableListener.exec(RunExecutableListener.java:70)
        at
org.apache.solr.core.RunExecutableListener.postCommit(RunExecutableListener.java:97)
        at
org.apache.solr.update.UpdateHandler.callPostCommitCallbacks(UpdateHandler.java:99)
        at
org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:514)
        at
org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestHandler.java:214)
        at
org.apache.solr.handler.XmlUpdateRequestHandler.handleRequestBody(XmlUpdateRequestHandler.java:84)
        at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:77)
        at org.apache.solr.core.SolrCore.execute(SolrCore.java:658)
        at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:191)
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:159)
        at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1089)
        at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:365)
        at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
        at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:285)
        at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
        at org.mortbay.jetty.HttpConnection
$RequestHandler.content(HttpConnection.java:835)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:723)
        at
org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:202)
        at
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
        at org.mortbay.jetty.bio.SocketConnector
$Connection.run(SocketConnector.java:226)
        at org.mortbay.thread.BoundedThreadPool
$PoolThread.run(BoundedThreadPool.java:442)
Caused by: java.io.IOException: java.io.IOException: error=12, Cannot
allocate memory
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
        at java.lang.ProcessImpl.start(ProcessImpl.java:65)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
        ... 28 more


        We have 3 Solr Servers, one Master and 2 Slaves, there is a Cron
Job doing Commits every 5 minutes on Master with Inserts/Deletes and
Slaves Rsync last Index Version from Master every 10 minutes, all 3
Servers have this Setup :

        * RHEL 5.0 OS 64 bits
        * 16 GB RAM Memory 

        We are giving Solr Java process 10 GB RAM Memory, did anybody
face this error ? Snapshooter process will try to allocate memory from
the 6GB RAM Memory available for the OS ? should we reduce 10GB to 8GB
for Solr Java process ?

        Thanks for your help


        Pablo