Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-22 Thread Umesh Prasad
Hi Shawn, Thanks for the advice :). The JVM heap Size usage on indexer machine has been consistency about 95% (both total and old gen) for past 3 days. It might have nothing to do with Solr 3.6 Vs solr 4.2 .. Because Solr 3.6 indexer gets restarted once in 2-3 days. Will investigate why

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Shawn Heisey
On 5/21/2013 9:22 PM, Umesh Prasad wrote: > This is our own implementation of data source (canon name > com.flipkart.w3.solr.MultiSPCMSProductsDataSource) , which pulls the data > from out downstream service and it doesn't cache data in RAM. It fetches > the data in batches of 200 and iterates

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Umesh Prasad
Hi Shawn, This is our own implementation of data source (canon name com.flipkart.w3.solr.MultiSPCMSProductsDataSource) , which pulls the data from out downstream service and it doesn't cache data in RAM. It fetches the data in batches of 200 and iterates over it when DIH asks for it. I will che

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Shawn Heisey
On 5/21/2013 5:14 PM, Umesh Prasad wrote: We have sufficient RAM on machine ..64 GB and we have given JVM 32 GB of memory. The machine runs Indexing primarily. The JVM doesn't run out of memory. It is the particular IndexWriterSolrCore which has .. May be we have specified too low a memory for I

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Umesh Prasad
on a machine with more memory. Or did you do that already? > > -- Jack Krupansky > > -Original Message- From: Umesh Prasad > Sent: Tuesday, May 21, 2013 1:57 AM > To: solr-user@lucene.apache.org > Subject: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1 >

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Jack Krupansky
Try again on a machine with more memory. Or did you do that already? -- Jack Krupansky -Original Message- From: Umesh Prasad Sent: Tuesday, May 21, 2013 1:57 AM To: solr-user@lucene.apache.org Subject: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1 Hi All, I am hitting

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Shawn Heisey
On 5/20/2013 11:57 PM, Umesh Prasad wrote: >I am hitting an OOM error while trying to do an hard commit on one of > the cores. > > Transaction log dir is Empty and DIH shows indexing going on for 13 hrs.. > > *Indexing since 13h 22m 22s* > Requests: 5,211,392 (108/s), Fetched: 1,902,792 (40/

Re: Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-21 Thread Otis Gospodnetic
Hi, Maybe you van share more info, such as your java command line or jstat output from right before the oom ... Otis Solr & ElasticSearch Support http://sematext.com/ On May 21, 2013 1:58 AM, "Umesh Prasad" wrote: > Hi All, >I am hitting an OOM error while trying to do an hard commit on one

Hard Commit giving OOM Error on Index Writer in Solr 4.2.1

2013-05-20 Thread Umesh Prasad
Hi All, I am hitting an OOM error while trying to do an hard commit on one of the cores. Transaction log dir is Empty and DIH shows indexing going on for 13 hrs.. *Indexing since 13h 22m 22s* Requests: 5,211,392 (108/s), Fetched: 1,902,792 (40/s), Skipped: 106,853, Processed: 1,016,696 (21/s)