On Mon, Feb 7, 2011 at 9:29 PM, Mark <static.void....@gmail.com> wrote: > I'm receiving the following exception when trying to perform a full-import > (~30 hours). Any idea on ways I could fix this? > > Is there an easy way to use DIH to break apart a full-import into multiple > pieces? IE 3 mini-imports instead of 1 large import? > > Thanks. > > > > > Feb 7, 2011 5:52:33 AM org.apache.solr.handler.dataimport.JdbcDataSource > closeConnection > SEVERE: Ignoring Error when closing connection > com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: > Communications link failure during rollback(). Transaction resolution > unknown. [...]
This looks like a network issue, or some other failure in communicating with the mysql database. Is that a possibility? Also, how many records are you importing, what is the data size, what is the quality of the network connection, etc.? One way to break up the number of records imported at a time is to shard your data at at the database level, but the advisability of this option depends on whether there is a more fundamental issue. Regards, Gora
