On 4/3/2017 9:55 AM, Islam Omar wrote: > I have a trouble problem when doing *full import in solr 6.4.1 using MySQL > DB , the problem is : * > > i need to create 1 core which will be around 9,500,000 documents , when i > do full import with *batchSize* *= -1* in datasource , everything was ok > but when the solr finish fetching data from database it *can't stop* running > full import command and continue trying to fetch another data. > > *the log throws this exception * > > Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: > java.sql.SQLException: Operation not allowed after ResultSet closed <snip> > Caused by: java.sql.SQLException: Operation not allowed after ResultSet > closed <snip> > org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.hasnext(JdbcDataSource.java:458)
This error is happening in code added by SOLR-2199, in version 6.1.0: https://issues.apache.org/jira/browse/SOLR-2199 Based on the error message and what I know about JDBC, I believe that it is isolated to the specific new code added by that issue. I have done DIH imports from MySQL with version 6.3.0, creating indexes much larger than your 9.5 million rows, without any problems. This is the first time I've seen this particular problem mentioned. If it WERE a bug in the SOLR-2199 code, it seems like it would have already been noticed in the nine months since 6.1.0 was released. It is more likely that there's an issue with the MySQL JDBC driver, or possibly a result of the specific SQL statements that are being used in the import. It is always possible that a new bug was introduced in 6.4.x, but DIH is heavily used by Solr users, and 6.4.x has been out for some time, so I think that a general problem with DIH would have already been reported at least once. At this point, I think Solr probably does not have a bug, but I'm not completely ruling that possibility out. What version of the MySQL JDBC driver are you using with Solr? I would strongly recommend that you use the latest GA version. That is currently version 5.1.41. There are development versions of the 6.0 driver available, but I do not recommend using those except in test environments where you can make changes without affecting production systems. https://dev.mysql.com/downloads/connector/j/5.1.html If a new driver doesn't help, we can look deeper into whether Solr has a bug. You may need to share your dataimport config, schema, and solrconfig.xml, after redacting any sensitive information. Thanks, Shawn
