On Thursday 10 September 2009 08:13:33 am Dan A. Dickey wrote:
> I'm posting documents to Solr using http (curl) from
> C++/C code and am seeing approximately 3.3 - 3.4
> documents per second being posted. Is this to be expected?
> Granted - I understand that this depends somewhat on the
> machine running Solr. By the way - I'm running Solr inside JBoss.
>
> I was hoping for maybe 20 or more docs/sec, and 3 or so
> is quite a way from that.
>
> Also, I'm posting just a single document at a time. I once tried
> 5 processes each posting documents, and that slowed things
> down considerably. Down into the multiple (5-10) seconds per document.
>
> Does anyone have suggestions on what I can try? I'll soon
> have better servers installed and will be splitting the indexing
> work from the searching - but at this point in time, I wasn't doing
> indexing while searching anyway. Thanks for any and all help!
Ok, I spent some time on this problem this morning, and have some
interesting results to share. I started off by making sure both boxes
were attached to the same switch - they weren't, but now are.
It didn't help.
I added some timing code... and found indeed that I was getting about
3.3 - 3.4 documents per second to index. Not so good.
I stopped JBoss (and Solr) and built up a version of the example
stuff that would run my current configuration instead of the example.
Reading the documentation - this runs Solr in a Jetty container.
And this resulted in indexing speeds ranging between 20 - 30 documents
per second. Much more acceptable. And also, with a quick test of using
two processes to index - I hit a rate of about 37 dps. Much nicer.
I don't know yet how this actually scales - but I intend to find out.
We've almost got some nice quad core xeon's ready...
Our JBoss expert and I will be looking into why this might be occurring.
Does anyone know of any JBoss related slowness with Solr?
And does anyone have any other sort of suggestions to speed indexing
performance? Thanks for your help all! I'll keep you up to date with
further progress.
-Dan
--
Dan A. Dickey | Senior Software Engineer
Savvis
10900 Hampshire Ave. S., Bloomington, MN 55438
Office: 952.852.4803 | Fax: 952.852.4951
E-mail: dan.dic...@savvis.net
