Re: DIH - Out of Memory error when using CachedsqlEntityProcessor

2010-03-13 Thread Mark Miller
I don't really follow DataImportHandler, but it looks like its using an unbounded cache (simple HashMap). Perhaps we should make the cache size configurable? The impl seems a little odd - the caching occurs in the base class - so caching impls that extends it don't really have full control - t

Re: DIH - Out of Memory error when using CachedsqlEntityProcessor

2010-03-13 Thread JavaGuy84
Erik, I have seen many posts regarding out of memory error but I am not sure whether they are using cachesqlEntityProcessor.. I want to know if there is a way to flush out the buffer of cache instead of storing everything in cache. I can clearly see the heapsize growing like anything if I use

Re: DIH - Out of Memory error when using CachedsqlEntityProcessor

2010-03-13 Thread Erick Erickson
Have you searched the users' list? This question has come up multiple times and you'll find your question has probably already been answered. Let us know if you come up blank... Best Erick On Sat, Mar 13, 2010 at 3:56 PM, JavaGuy84 wrote: > > Sorry forgot to attach the error log, > > Error Log:

Re: DIH - Out of Memory error when using CachedsqlEntityProcessor

2010-03-13 Thread JavaGuy84
Sorry forgot to attach the error log, Error Log: - org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.OutOfMe moryError: Java heap space at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde r.java:650) at org.apache.solr.handler.da