Re: More heap usage in Solr during indexing

2014-03-19 Thread solr2020
We are doing Autocommit for every five minutes. -- View this message in context: http://lucene.472066.n3.nabble.com/More-heap-usage-in-Solr-during-indexing-tp4124898p4125497.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: More heap usage in Solr during indexing

2014-03-17 Thread solr2020
Yes Shawn. our data source is oracle DB. Here is the datasource section config. -- View this message in context: http://lucene.472066.n3.nabble.com/More-heap-usage-in-Solr-during-indexing-tp4124898p4124934.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: More heap usage in Solr during indexing

2014-03-17 Thread Shawn Heisey
On 3/17/2014 12:39 PM, solr2020 wrote: previously we faced OOM when we try to index 1.2M records at the same time. Now we divided that into two chunks and indexing twice. So now we are not getting OOM but heap usage is more. So we are analyzing and trying to find the cause to make sure we shouldn

Re: More heap usage in Solr during indexing

2014-03-17 Thread solr2020
previously we faced OOM when we try to index 1.2M records at the same time. Now we divided that into two chunks and indexing twice. So now we are not getting OOM but heap usage is more. So we are analyzing and trying to find the cause to make sure we shouldn't get OOM again. -- View this message

Re: More heap usage in Solr during indexing

2014-03-17 Thread Greg Walters
It's entirely possible that you're seeing higher memory usage while indexing due to more objects being created and abandoned. Another thing to consider could be your commit settings. Perhaps http://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html can answer some of your q

Re: More heap usage in Solr during indexing

2014-03-17 Thread Greg Walters
Are your JVM running out of ram (actual exceptions) or is the used heap just reaching 16G prior to a garbage collection? If it's the later then that is expected behavior and is how Java's garbage collection works. Thanks, Greg On Mar 17, 2014, at 1:26 PM, solr2020 wrote: > Hi, > > we have 80