Re: ExtractingRequestHandler causes Out of Memory Error

2012-10-03 Thread Jan Høydahl
Hi, If you like, you can open a JIRA issue on this and provide as much info as possible. Someone can then look into (potential) memory optimization of this part of the code. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com Solr Training - www.solrtraining.com 28. sep.

Re: ExtractingRequestHandler causes Out of Memory Error

2012-09-27 Thread Shigeki Kobayashi
Hi Jan. Thank you very much for your advice. So I understand Solr needs more memory to parse the files. To parse a file of size x, it needs double memory (2x). Then how much memory allocation should be taken to heap size? 8x? 16x? Regards, Shigeki 2012/9/28 Jan Høydahl > Please try to incr

Re: ExtractingRequestHandler causes Out of Memory Error

2012-09-27 Thread Jan Høydahl
Please try to increase -Xmx and see how much RAM you need for it to succeed. I believe it is simply a case where this particular file needs double memory (480Mb) to parse and you have only allocated 1Gb (which is not particularly much). Perhaps the code could be optimized to avoid the Arrays.cop

Re: ExtractingRequestHandler causes Out of Memory Error

2012-09-27 Thread Lance Norskog
ot; | To: solr-user@lucene.apache.org | Sent: Thursday, September 27, 2012 2:22:06 AM | Subject: ExtractingRequestHandler causes Out of Memory Error | | Hi guys, | | | I use Manifold CF to crawl files in Windows file server and index | them to | Solr using Extracting Request Handler. | Most of the docum

ExtractingRequestHandler causes Out of Memory Error

2012-09-27 Thread Shigeki Kobayashi
Hi guys, I use Manifold CF to crawl files in Windows file server and index them to Solr using Extracting Request Handler. Most of the documents are succesfully indexed but some are failed and Out of Memory Error occurs in Solr, so I need some advice. Those failed files are not so big and they ar