This is also what I have done, but I agree with the notion of using something external to load the data.
-----Original Message----- From: Dyer, James [mailto:james.d...@ingramcontent.com] Sent: Thursday, October 15, 2015 9:24 AM To: solr-user@lucene.apache.org Subject: RE: DIH parallel processing Nabil, What we do is have multiple dih request handlers configured in solrconfig.xml. Then in the sql query we put something like "where mod(id, ${partition})=0". Then an external script calls a full import on each request handler at the same time and monitors the response. This isn't the most elegant solution but it gets around the fact that DIH is single-threaded. James Dyer Ingram Content Group -----Original Message----- From: nabil Kouici [mailto:koui...@yahoo.fr] Sent: Thursday, October 15, 2015 3:58 AM To: Solr-user Subject: DIH parallel processing Hi All, I'm using DIH to index more than 15M from Sql Server to Solr. This take more than 2 hours. Big amount of this time is consumed by data fetching from database. I'm thinking about a solution to have parallel (thread) loud in the same DIH. Each thread load a part of data. Do you have any experience with this kind of situation? Regards,Nabil.
