Have you searched the users' list? This question has come up multiple times and you'll find your question has probably already been answered. Let us know if you come up blank...
Best Erick On Sat, Mar 13, 2010 at 3:56 PM, JavaGuy84 <bbar...@gmail.com> wrote: > > Sorry forgot to attach the error log, > > Error Log: > --------- > org.apache.solr.handler.dataimport.DataImportHandlerException: > java.lang.OutOfMe > moryError: Java heap space > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde > r.java:650) > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde > r.java:605) > at > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j > ava:261) > at > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java > :185) > at > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo > rter.java:333) > at > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j > ava:391) > at > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja > va:372) > Caused by: java.lang.OutOfMemoryError: Java heap space > at java.util.HashMap.<init>(Unknown Source) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.g > etARow(JdbcDataSource.java:281) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.a > ccess$800(JdbcDataSource.java:228) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1 > .next(JdbcDataSource.java:266) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1 > .next(JdbcDataSource.java:269) > at > org.apache.solr.handler.dataimport.CachedSqlEntityProcessor.getAllNon > CachedRows(CachedSqlEntityProcessor.java:70) > at > org.apache.solr.handler.dataimport.EntityProcessorBase.getIdCacheData > (EntityProcessorBase.java:194) > at > org.apache.solr.handler.dataimport.CachedSqlEntityProcessor.nextRow(C > achedSqlEntityProcessor.java:58) > at > org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent > ityProcessorWrapper.java:233) > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde > r.java:579) > ... 6 more > Mar 13, 2010 3:52:09 PM org.apache.solr.handler.dataimport.DataImporter > doFullIm > port > SEVERE: Full Import failed > org.apache.solr.handler.dataimport.DataImportHandlerException: > java.lang.OutOfMe > moryError: Java heap space > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde > r.java:650) > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde > r.java:605) > at > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.j > ava:261) > at > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java > :185) > at > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImpo > rter.java:333) > at > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.j > ava:391) > at > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.ja > va:372) > Caused by: java.lang.OutOfMemoryError: Java heap space > at java.util.HashMap.<init>(Unknown Source) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.g > etARow(JdbcDataSource.java:281) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.a > ccess$800(JdbcDataSource.java:228) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1 > .next(JdbcDataSource.java:266) > at > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1 > .next(JdbcDataSource.java:269) > at > org.apache.solr.handler.dataimport.CachedSqlEntityProcessor.getAllNon > CachedRows(CachedSqlEntityProcessor.java:70) > at > org.apache.solr.handler.dataimport.EntityProcessorBase.getIdCacheData > (EntityProcessorBase.java:194) > at > org.apache.solr.handler.dataimport.CachedSqlEntityProcessor.nextRow(C > achedSqlEntityProcessor.java:58) > at > org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(Ent > ityProcessorWrapper.java:233) > at > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilde > r.java:579) > ... 6 more > Mar 13, 2010 3:52:09 PM org.apache.solr.update.DirectUpdateHandler2 > rollback > INFO: start rollback > Exception in thread "Timer-0" java.lang.OutOfMemoryError: Java heap space > at java.util.ArrayList.<init>(Unknown Source) > at java.util.ArrayList.<init>(Unknown Source) > at org.mortbay.util.Scanner.reportDifferences(Scanner.java:289) > at org.mortbay.util.Scanner.scan(Scanner.java:254) > at org.mortbay.util.Scanner$1.run(Scanner.java:225) > at java.util.TimerThread.mainLoop(Unknown Source) > Exception in thread "SocketServerInstance-crprdnii1soe-31000" at > java.util.Tim > erThread.run(Unknown Source) > java.lang.OutOfMemoryError: Java heap space > at java.nio.CharBuffer.wrap(Unknown Source) > at sun.nio.cs.StreamEncoder.implWrite(Unknown Source) > at sun.nio.cs.StreamEncoder.write(Unknown Source) > at java.io.OutputStreamWriter.write(Unknown Source) > at java.io.BufferedWriter.flushBuffer(Unknown Source) > at java.io.PrintStream.write(Unknown Source) > at java.io.PrintStream.print(Unknown Source) > at > com.metamatrix.common.comm.platform.socket.PrintStreamSocketLog.log(P > rintStreamSocketLog.java:169) > at > com.metamatrix.common.comm.platform.socket.PrintStreamSocketLog.log(P > rintStreamSocketLog.java:175) > at > com.metamatrix.common.comm.platform.socket.PrintStreamSocketLog.logEr > ror(PrintStreamSocketLog.java:71) > at > com.metamatrix.common.comm.platform.socket.client.SocketServerInstanc > eImpl$1.run(SocketServerInstanceImpl.java:578) > at java.lang.Thread.run(Unknown Source) > Mar 13, 2010 3:52:09 PM org.apache.solr.update.DirectUpdateHandler2 > rollback > INFO: end_rollback > > Thanks, > Barani > > JavaGuy84 wrote: > > > > Hi, > > > > I am using CachedsqlEntityProcessor in my DIH dataconfig to reduce the > > number of queries executed against the database , > > > > <Entity1 query="select * from x" processor= "CachedsqlEntityProcessor"/> > > <Entity2 query="select * from y" processor= "CachedsqlEntityProcessor" > > cachekey="id" cachelookup="x.id"/> > > > > > > I having more than 2 million rows returned for Entity 2 and around 300000 > > rows returned for entity1. > > > > I am have set the heap size to 1 GB but even then I am always getting > heap > > out of size error. I am not sure how to flush the documents in buffer at > > certain condition. I tried to enable Autocommit / reduced the > > maxdocbuffersize but of no use.. Can someone let me know what is the best > > way to overcome this issue? > > > > Thanks, > > Barani > > > > -- > View this message in context: > http://old.nabble.com/DIH---Out-of-Memory-error-when-using-CachedsqlEntityProcessor-tp27889623p27890751.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
