On 9/16/2013 8:26 PM, YouPeng Yang wrote: > I'm using the DIH to import data from oracle database with Solr4.4 > Finally I get 2.7GB index data and 4.1GB tlog data.And the number of > docs was 10900000. > > At first, I move the 2.7GB index data to another new Solr Server in > tomcat7. After I start the tomcat ,I find the total number of docs was just > half of the orginal number. > So I thought that maybe the left docs were not commited to index > files,and the tlog needed to be replayed .
You need to turn on autoCommit in your solrconfig.xml so that there are hard commits happening on a regular basis that flush all indexed data to disk and start new transaction log files. I will give you a link with some information about that below. > Sequently , I moved the 2.7GB index data and 4.1GB tlog data to the new > Solr Server in tomcat7. > After I start the tomcat,an exception comes up as [1]. > Then it halts.I can not access the tomcat server URL. > I noticed that the CPU utilization was high by using the comand: top > -d 1 | grep tomcatPid. > I thought solr was replaying the updatelog.And I wait a long time and it > still was replaying. As results ,I give up. I don't know what the exception was about, but it is likely that it WAS replaying the log. With 4.1GB of transaction log, that's going to take a LONG time, during which Solr will be unavailable. It always replays the entire transaction log. The key, as mentioned above, is in keeping that log small. Here's a wiki page about the slow startup problem and an example of how to configure autoCommit to deal with it: http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup There's a lot of other good information on that page. Thanks, Shawn
