Maybe your rate of update is so high that the commit never gets a chance to
run. So, maybe all these uncommitted updates are buffered up and using
excess memory.
Try explicit commits from SolrJ, but less frequently. Or maybe if you just
pause your updates periodically (every 30 seconds or so) the auto-commit
would get a chance to occur. Although I have no idea how long a pause might
be needed.
-- Jack Krupansky
-----Original Message-----
From: Nick Koton
Sent: Sunday, July 15, 2012 10:52 AM
To: solr-user@lucene.apache.org ; yo...@lucidimagination.com
Subject: RE: SOLR 4 Alpha Out Of Mem Err
Do you have the following hard autoCommit in your config (as the stock
server does)?
<autoCommit>
<maxTime>15000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
I have tried with and without that setting. When I described running with
auto commit, that setting is what I mean. I have varied the time in the
range 10,000-60,000 msec. I have tried this setting with and without soft
commit in the server config file.
I have tried without this setting, but specifying the commit within time in
the solrj client in the add method.
In both these cases, the client seems to overrun the server and out of
memory in the server results. One clarification I should make is that after
the server gets out of memory, the solrj client does NOT receive an error.
However, the documents indexed do not reliably appear to queries.
Approach #3 is to remove the autocommit in the server config, issue the add
method without commit within, but issue commits in the solrj client with
wait for sync and searcher set to true. In case #3, I do not see the out of
memory in the server. However, document index rates are restricted to about
1,000 per second.
Nick
-----Original Message-----
From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley
Sent: Sunday, July 15, 2012 5:15 AM
To: solr-user@lucene.apache.org
Subject: Re: SOLR 4 Alpha Out Of Mem Err
Do you have the following hard autoCommit in your config (as the stock
server does)?
<autoCommit>
<maxTime>15000</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
This is now fairly important since Solr now tracks information on every
uncommitted document added.
At some point we should probably hardcode some mechanism based on number of
documents or time.
-Yonik
http://lucidimagination.com
