First, did you check the Solr log on the server that’s getting the docs to index? Sometimes there’s a better error message/stack trace there.
Second, although people do use DIH for production, I tend to think of it as a PoC tool. You can pretty easily write a SolrJ program that does the import, and that gives you a lot more control. Here;s a sample of doing that: https://lucidworks.com/post/indexing-with-solrj/ Best, Erick > On May 18, 2020, at 8:30 AM, Eric DEWITTE <eric.dewi...@iptwins.com> wrote: > > Hello, > > I need help because I'm stuck on a data import. > > I have an old server under solr 4.6 and I wanted to import data from it to a > new server 8.4 > > The old server has a pretty big core with 352 million records... > > I managed to import in increments of 100,000 up to 169 million, but since > then I've been getting these error messages: > Exception in thread "main" org.apache.solr.client.solrj.SolrServerException: > Timeout occurred while waiting response from server at: > http://127.0.0.1:8984/solr/core1 > at > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:676) > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265) > at > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248) > at > org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211) > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1003) > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1018) > at Migrator.main(Migrator.java:43) > Caused by: java.net.SocketTimeoutException: Read timed out > > I tried to lower the slice (50,000 instead of 100,000) but same effects. > > I don't know where I can go to increase the timeout or if I have to do > something else to be able to continue and finish my import. > > Thank you > > E.D.
