Oh and one more thing...For historical reasons our apps run using msft
technologies, so using SolrJ would be next to impossible at the present
time....
Thanks in advance for your help!
-- Bill
--------------------------------------------------
From: "William Pierce" <evalsi...@hotmail.com>
Sent: Saturday, October 10, 2009 5:47 PM
To: <solr-user@lucene.apache.org>
Subject: Tips on speeding up indexing needed...
Folks:
I have a corpus of approx 6 M documents each of approx 4K bytes.
Currently, the way indexing is set up I read documents from a database and
issue solr post requests in batches (batches are set up so that the
maxPostSize of tomcat which is set to 2MB is adhered to). This means that
in each batch we write approx 600 or so documents to SOLR. What I am
seeing is that I am able to push about 2500 docs per minute or approx 40
or so per second.
I saw in Erik's talk on Friday that speeds of 250 docs/sec to 25000
docs/sec have been achieved. Needless to say I am sure that performance
numbers vary widely and are dependent on the domain, machine
configurations, etc.
I am running on Windows 2003 server, with 4 GB RAM, dual core xeon.
Any tips on what I can do to speed this up?
Thanks,
Bill
