No didn't double post, my be it was in my outbox and went out again. The queries outside solr dont take so long, to return around 500000 rows it takes 250 seconds, so I am doing a delta import of around 500,000 rows at a time. I have tried turning auto commit on and things are moving a bit faster now. Are there any more tweeking i can do?
Also, planning to move to master-salve model, but am failing to understand where to start exactly. Regards, Rohit ________________________________ From: lee carroll <lee.a.carr...@googlemail.com> To: solr-user@lucene.apache.org Sent: Sun, 5 June, 2011 4:59:44 AM Subject: Re: URGENT HELP: Improving Solr indexing time Rohit - you have double posted maybe - did Otis's answer not help with your issue or at least need a response to clarify ? On 4 June 2011 22:53, Chris Cowan <chrisco...@plus3network.com> wrote: > How long does the query against the DB take (outside of Solr)? If that's slow >then it's going to take a while to update the index. You might need to figure >a >way to break things up a bit, maybe use a delta import instead of a full >import. > > Chris > > On Jun 4, 2011, at 6:23 AM, Rohit Gupta wrote: > >> My Solr server takes very long to update index. The table it hits to index is >> huge with 10Million + records , but even in that case I feel this is very long >> time to index. Below is the snapshot of the /dataimport page >> >> <str name="status">busy</str> >> <str name="importResponse">A command is still running...</str> >> <lst name="statusMessages"> >> <str name="Time Elapsed">1:53:39.664</str> >> <str name="Total Requests made to DataSource">16276</str> >> <str name="Total Rows Fetched">24237</str> >> <str name="Total Documents Processed">16273</str> >> <str name="Total Documents Skipped">0</str> >> <str name="Full Dump Started">2011-06-04 11:25:26</str> >> </lst> >> >> How can i determine why this is happening and how can I improve this. During >>all >> our test on the local server before the migration we could index 5 million >> records in 4-5 hrs, but now its taking too long on the live server. >> >> Regards, >> Rohit > >
