Yes, initially I figured that we are accidentally re-using a closed data source. But Noble has pinned it right. I guess you can try looking into your JDBC driver's documentation for a setting which increases the connection alive-ness.
On Mon, Jan 5, 2009 at 5:29 PM, Noble Paul നോബിള് नोब्ळ् < noble.p...@gmail.com> wrote: > I guess the indexing of a doc is taking too long (may be because of > the de-dup patch) and the resultset gets closed automaticallly (timed > out) > --Noble > > On Mon, Jan 5, 2009 at 5:14 PM, Marc Sturlese <marc.sturl...@gmail.com> > wrote: > > > > Donig this fix I get the same error :( > > > > I am going to try to set up the last nigthly build... let's see if I have > > better luck. > > > > The thing is it stop indexing at the doc num 150.000 aprox... and give me > > that mysql exception error... Without DeDuplication patch I can index 2 > > milion docs without problems... > > > > I am pretty lost with this... :( > > > > > > Shalin Shekhar Mangar wrote: > >> > >> Yes I meant the 05/01/2008 build. The fix is a one line change > >> > >> Add the following as the last line of DataConfig.Entity.clearCache() > >> dataSrc = null; > >> > >> > >> > >> On Mon, Jan 5, 2009 at 4:22 PM, Marc Sturlese > >> <marc.sturl...@gmail.com>wrote: > >> > >>> > >>> Shalin you mean I should test the 05/01/2008 nighlty? maybe with this > one > >>> works? If the fix you did is not really big can u tell me where in the > >>> source is and what is it for? (I have been debuging and tracing a lot > the > >>> dataimporthandler source and I I would like to know what the > imporovement > >>> is > >>> about if it is not a problem...) > >>> > >>> Thanks! > >>> > >>> > >>> Shalin Shekhar Mangar wrote: > >>> > > >>> > Marc, I've just committed a fix which may have caused the bug. Can > you > >>> use > >>> > svn trunk (or the next nightly build) and confirm? > >>> > > >>> > On Mon, Jan 5, 2009 at 3:10 PM, Noble Paul നോബിള് नोब्ळ् < > >>> > noble.p...@gmail.com> wrote: > >>> > > >>> >> looks like a bug w/ DIH with the recent fixes. > >>> >> --Noble > >>> >> > >>> >> On Mon, Jan 5, 2009 at 2:36 PM, Marc Sturlese > >>> <marc.sturl...@gmail.com> > >>> >> wrote: > >>> >> > > >>> >> > Hey there, > >>> >> > I was using the Deduplication patch with Solr 1.3 release and > >>> >> everything > >>> >> was > >>> >> > working perfectly. Now I upgraded to a nigthly build (20th > december) > >>> to > >>> >> be > >>> >> > able to use new facet algorithm and other stuff and DeDuplication > is > >>> >> not > >>> >> > working any more. I have followed exactly the same steps to apply > >>> the > >>> >> patch > >>> >> > to the source code. I am geting this error: > >>> >> > > >>> >> > WARNING: Error reading data > >>> >> > com.mysql.jdbc.CommunicationsException: Communications link > failure > >>> due > >>> >> to > >>> >> > underlying exception: > >>> >> > > >>> >> > ** BEGIN NESTED EXCEPTION ** > >>> >> > > >>> >> > java.io.EOFException > >>> >> > > >>> >> > STACKTRACE: > >>> >> > > >>> >> > java.io.EOFException > >>> >> > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905) > >>> >> > at > >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2404) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) > >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) > >>> >> > at > >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) > >>> >> > at > >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) > >>> >> > at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) > >>> >> > > >>> >> > > >>> >> > ** END NESTED EXCEPTION ** > >>> >> > Last packet sent to the server was 202481 ms ago. > >>> >> > at > >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) > >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) > >>> >> > at > >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) > >>> >> > at > >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) > >>> >> > at com.mysql.jdbc.ResultSet.next(ResultSet.java:6144) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:294) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) > >>> >> > Jan 5, 2009 10:06:16 AM > >>> >> org.apache.solr.handler.dataimport.JdbcDataSource > >>> >> > logError > >>> >> > WARNING: Exception while closing result set > >>> >> > com.mysql.jdbc.CommunicationsException: Communications link > failure > >>> due > >>> >> to > >>> >> > underlying exception: > >>> >> > > >>> >> > ** BEGIN NESTED EXCEPTION ** > >>> >> > > >>> >> > java.io.EOFException > >>> >> > > >>> >> > STACKTRACE: > >>> >> > > >>> >> > java.io.EOFException > >>> >> > at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:1905) > >>> >> > at > >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2351) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) > >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) > >>> >> > at > >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) > >>> >> > at > >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) > >>> >> > at > >>> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150) > >>> >> > at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488) > >>> >> > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) > >>> >> > > >>> >> > > >>> >> > ** END NESTED EXCEPTION ** > >>> >> > > >>> >> > Last packet sent to the server was 202482 ms ago. > >>> >> > at > >>> com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2563) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2862) > >>> >> > at > com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:771) > >>> >> > at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1289) > >>> >> > at > >>> >> com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:362) > >>> >> > at > >>> com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:352) > >>> >> > at > >>> com.mysql.jdbc.RowDataDynamic.close(RowDataDynamic.java:150) > >>> >> > at com.mysql.jdbc.ResultSet.realClose(ResultSet.java:6488) > >>> >> > at com.mysql.jdbc.ResultSet.close(ResultSet.java:736) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.close(JdbcDataSource.java:312) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:304) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.access$400(JdbcDataSource.java:189) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator$1.hasNext(JdbcDataSource.java:225) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:229) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:76) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:351) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:193) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:144) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:334) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:407) > >>> >> > at > >>> >> > > >>> >> > >>> > org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:388) > >>> >> > > >>> >> > Mark do you know if I should to some extra hack to run > DeDuplication > >>> >> patch > >>> >> > in the new solr version (nightly)? Has something changed in the > >>> source > >>> >> that > >>> >> > make it not work or it's simply that i am doing something wrong? > >>> >> > > >>> >> > Thanks in advanced > >>> >> > -- > >>> >> > View this message in context: > >>> >> > >>> > http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21287327.html > >>> >> > Sent from the Solr - User mailing list archive at Nabble.com. > >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> --Noble Paul > >>> >> > >>> > > >>> > > >>> > > >>> > -- > >>> > Regards, > >>> > Shalin Shekhar Mangar. > >>> > > >>> > > >>> > >>> -- > >>> View this message in context: > >>> > http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21288939.html > >>> Sent from the Solr - User mailing list archive at Nabble.com. > >>> > >>> > >> > >> > >> -- > >> Regards, > >> Shalin Shekhar Mangar. > >> > >> > > > > -- > > View this message in context: > http://www.nabble.com/Deduplication-patch-not-working-in-nightly-build-tp21287327p21289643.html > > Sent from the Solr - User mailing list archive at Nabble.com. > > > > > > > > -- > --Noble Paul > -- Regards, Shalin Shekhar Mangar.
