Re: Solr out of memory exception

2012-03-15 Thread Lance Norskog
gt; >> On Mar 15, 2012, at 6:39 AM, Husain, Yavar wrote: >> >>> Thanks a ton. >>> ________________ >>> From: Li Li [fancye...@gmail.com] >>> Sent: Thursday, March 15, 2012 12:11 PM >>> To: Husain, Yavar >>>

Re: Solr out of memory exception

2012-03-15 Thread Erick Erickson
ton. >> >> From: Li Li [fancye...@gmail.com] >> Sent: Thursday, March 15, 2012 12:11 PM >> To: Husain, Yavar >> Cc: solr-user@lucene.apache.org >> Subject: Re: Solr out of memory exception >> >> it seems you ar

Re: Solr out of memory exception

2012-03-15 Thread François Schiettecatte
> From: Li Li [fancye...@gmail.com] > Sent: Thursday, March 15, 2012 12:11 PM > To: Husain, Yavar > Cc: solr-user@lucene.apache.org > Subject: Re: Solr out of memory exception > > it seems you are using 64bit jvm(32bit jvm can only allocate about 1.5GB). > you should ena

RE: Solr out of memory exception

2012-03-15 Thread Husain, Yavar
Thanks a ton. From: Li Li [fancye...@gmail.com] Sent: Thursday, March 15, 2012 12:11 PM To: Husain, Yavar Cc: solr-user@lucene.apache.org Subject: Re: Solr out of memory exception it seems you are using 64bit jvm(32bit jvm can only allocate about 1.5GB

Re: Solr out of memory exception

2012-03-15 Thread Li Li
- > From: "Li Li"; > Date: Thu, Mar 15, 2012 02:41 PM > To: "Husain, Yavar"; > Cc: "solr-user@lucene.apache.org"; > Subject: Re: Solr out of memory exception > > > it seems you are using 64bit jvm(32bit jvm can only allocate about

Re: Solr out of memory exception

2012-03-15 Thread C.Yunqin
why should enable pointer compression? -- Original -- From: "Li Li"; Date: Thu, Mar 15, 2012 02:41 PM To: "Husain, Yavar"; Cc: "solr-user@lucene.apache.org"; Subject: Re: Solr out of memory exception it seems you are

Re: Solr out of memory exception

2012-03-14 Thread Li Li
ver with exactly same system and solr configuration & > memory it is working fine? > > > -Original Message- > From: Li Li [mailto:fancye...@gmail.com] > Sent: Thursday, March 15, 2012 11:11 AM > To: solr-user@lucene.apache.org > Subject: Re: Solr out of memory excep

RE: Solr out of memory exception

2012-03-14 Thread Husain, Yavar
pace allocated but then how come on a different server with exactly same system and solr configuration & memory it is working fine? -Original Message- From: Li Li [mailto:fancye...@gmail.com] Sent: Thursday, March 15, 2012 11:11 AM To: solr-user@lucene.apache.org Subject: Re: Sol

Re: Solr out of memory exception

2012-03-14 Thread Li Li
how many memory are allocated to JVM? On Thu, Mar 15, 2012 at 1:27 PM, Husain, Yavar wrote: > Solr is giving out of memory exception. Full Indexing was completed fine. > Later while searching maybe when it tries to load the results in memory it > starts giving this exception. Though with the sam

Re: SOlR -- Out of Memory exception

2011-06-17 Thread jyn7
I commented the autocommit option and tried uploading the file (a smaller file now 5 million records) and I hit an oom again: Jun 17, 2011 2:32:59 PM org.apache.solr.common.SolrException log SEVERE: java.lang.OutOfMemoryError: Java heap space -- View this message in context: http://lucene.472066

Re: SOlR -- Out of Memory exception

2011-06-17 Thread Yonik Seeley
On Fri, Jun 17, 2011 at 1:30 AM, pravesh wrote: > If you are sending whole CSV in a single HTTP request using curl, why not > consider sending it in smaller chunks? Smaller chunks should not matter - Solr streams from the input (i.e. the whole thing is not buffered in memory). It could be relate

Re: SOlR -- Out of Memory exception

2011-06-17 Thread jyn7
I did that , but when I split them into 5 mill records, the first file went through fine, when I started processing the second file SOLR hit an OOM again: org.apache.solr.common.SolrException log SEVERE: java.lang.OutOfMemoryError: Java heap space at org.apache.lucene.index.FreqProxTermsWri

Re: SOlR -- Out of Memory exception

2011-06-16 Thread pravesh
If you are sending whole CSV in a single HTTP request using curl, why not consider sending it in smaller chunks? -- View this message in context: http://lucene.472066.n3.nabble.com/SOlR-Out-of-Memory-exception-tp3074636p3075091.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOlR -- Out of Memory exception

2011-06-16 Thread jyn7
Yes Eric, after changing the lock type to Single, I got an OOM after loading 5.5 million records. I am using the curl command to upload the csv. -- View this message in context: http://lucene.472066.n3.nabble.com/SOlR-Out-of-Memory-exception-tp3074636p3074765.html Sent from the Solr - User mailin

Re: SOlR -- Out of Memory exception

2011-06-16 Thread Erick Erickson
H, are you still getting your OOM after 7M records? Or some larger number? And how are you using the CSV uploader? Best Erick On Thu, Jun 16, 2011 at 9:14 PM, jyn7 wrote: > We just started using SOLR. I am trying to load a single file with 20 million > records into SOLR using the CSV uploade