On Apr 2, 2009, at 9:23 AM, Fergus McMenemie wrote:
Grant,
I should note, however, that the speed difference you are seeing may
not be as pronounced as it appears. If I recall during ApacheCon, I
commented on how long it takes to shutdown your Solr instance when
exiting it. That time it takes is in fact Solr doing the work that
was put off by not committing earlier and having all those deletes
pile up.
I am confused about "work that was put off" vs committing. My script
was doing a commit right after the CVS import, and you are right
about the massive times required to shut tomcat down. But in my tests
the time taken to do the commit was under a second, yet I had to allow
300secs for tomcat shutdown. Also I dont have any duplicates. So
what sort of work was being done at shutdown that was not being done
by a commit? Optimise!
The work being done is addressing the deletes, AIUI, but of course
there are other things happening during shutdown, too.
How long is the shutdown if you do a commit first and then a shutdown?
At any rate, I don't know that there is a satisfying answer to the
larger issue due to the things like the fsync stuff, which is an
overall win for Lucene/Solr despite it being more slower. Have you
tried running the tests on other machines (non-Mac?)
