It is entirely possible that the server is asking solr to shutdown. I'll have to ask the admin. I'm running Solr-1.4 inside of Jetty. I definitely have enough disk space. I think I did notice solr shutting down while it was idle. I just disregarded it as a fluke... Perhaps there's something going on. I will try to run this inside of tomcat and see what happens.
Not sure if this is related but I had to change the <lockType> to single instead of the default "native". With native, I get a lock time out when starting up solr. I also have <maxDocs> set to 10000. I did not want to have millions of uncommitted docs. I'm running under Linux RedHat. Regarding speed, the first million or so documents is done very quickly (maybe 3 hrs) but after that, things slows down tremendously. Thanks for the advice regarding solrj. I'll definitely look into that. P. On Mon, Nov 29, 2010 at 2:39 PM, Erick Erickson <erickerick...@gmail.com>wrote: > You're right, the OS is asking the server to shut down. In the default > example under Jetty, this is a result of issuing a crtl-c. Is it possible > that something is asking your server to quit? What servlet container > are you running under? Does the Solr server run for more than this > period if you're NOT indexing? And are you sure you have enough > resources, especially disk space? > > On another note, I'm surprised that it's taking 2 days to index 5m > documents. > That's less than 30 docs/second and Solr should handle a considerably > greater load than that. For whatever that's worth... > > And what version of Solr are you using? You may want to consider > writing something in SolrJ to do your indexing, it'll provide you more > flexible control over indexing than DIH.. > > Best > Erick > > On Mon, Nov 29, 2010 at 1:20 PM, Phong Dais <phong.gd...@gmail.com> wrote: > > > Hi, > > > > I am in the process of trying to index about 50 mil documents using the > > data > > import handler. > > For some reason, about 2 days into the import, I see this message > "shutdown > > hook executing" in the log and the solr web server instance exits > > "gracefully". > > I do not see any errors in the entire log. This has happened twice now, > > usually 5 mil or so documents into the import process. > > > > Does anyone out there knows what this message mean? It's an INFO log > > message so I don't think it is caused by any error. > > Does this problem occur because the os is asking the server to shut down > > (for whatever reason) or is there something wrong with the server causing > > it > > to shutdown? > > > > Thanks for any help, > > Phong > > >
