There is a known problem (that I can't find at the moment) where an uploaded file is retained while the next one is processed. When these two successive files are both huge, the coexistence of two giant causes an OOM.
Do you have this problem on the first file, second file, or at some time later? But, yes, a Content-Length during upload is obviously a great help. On Thu, May 13, 2010 at 5:39 AM, Grant Ingersoll <gsing...@apache.org> wrote: > > On May 12, 2010, at 1:58 PM, Christopher Baird wrote: > >> We're running into an out of memory problem when sending a large file to our >> SOLR server using the ContentStreamUpdateRequest. It appears that this >> happens because when the request method of CommonsHttpSolrServer is called >> (this is called even when using a StreamingUpdateSolrServer instance because >> the ContentStreamUpdateRequest class is not an instance of UpdateRequest) an >> InputStreamRequestEntity is used in the PostMethod buffers the content. The >> buffering happens because the content length is not provided and thus >> defaults to "CONTENT_LENGHT_AUTO" which instructs InputStreamRequestEntity >> to buffer the entire content. >> >> >> >> Is there an existing work-around to this? >> >> >> >> If not, can anyone think of why I wouldn't want to update the code to pass >> in the content-length and avoid the buffering (I don't want to walk down a >> path to find out I really stepped in something). > > I can't think of any reason not to put up a patch for it. -- Lance Norskog goks...@gmail.com
