Hi,
we are experiencing problems with the DataImportHandler in Solr 1.4.0. We are
reading datasets from MySQL and this worked fine before, but it seems the
DataImportHandler ran into an infinite loop and is requesting millions of data
rows from MySQL. This resulted in a High CPU Load on our DB Server.
Calling the dataimporthandler via curl with "abort" and doing a "full-import"
solved the issue, but before I tried that after restarting Tomcat/Solr DIH
seemed to be stuck in this infinite loop every time it came back/started.
I think these Log Messages are related to the problem:
SEVERE: The web application [/solr] created a ThreadLocal with key of type
[org.apache.solr.handler.dataimport.DataImporter$2] (value
[org.apache.solr.handler.dataimport.DataImporter$2@54b435c8]) and a value of
type [java.util.concurrent.atomic.AtomicLong] (value [580601]) but failed to
remove it when the web application was stopped. This is very likely to create a
memory leak.
org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/solr] appears to have started a thread named
[MySQL Statement Cancellation Timer] but has failed to stop it. This is very
likely to create a memory leak.
Has somebody encountered a similar behaviour before?, is there a way to prevent
DIH to run into these issues?
Our Current Setup:
Solr 1.4.0
apache-solr-dataimporthandler-1.4.1-dev
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Apache Tomcat/6.0.35
MySQL 5.1.50
Kind Regards,
Christian
