Thanks Erick, you confirmed my impressions! Thank you very much for the insights, an other opinion is welcome :)
Cheers 2015-10-05 14:55 GMT+01:00 Erick Erickson <erickerick...@gmail.com>: > SolrJ tends to be faster for several reasons, not the least of which > is that it sends packets to Solr in a more efficient binary format. > > Batching is critical. I did some rough tests using SolrJ and sending > docs one at a time gave a throughput of < 400 docs/second. > Sending 10 gave 2,300 or so. Sending 100 at a time gave > over 5,300 docs/second. Curiously, 1,000 at a time gave only > marginal improvement over 100. This was with a single thread. > YMMV of course. > > CloudSolrClient is definitely the better way to go with SolrCloud, > it routes the docs to the correct leader instead of having the > node you send the docs to do the routing. > > Best, > Erick > > On Mon, Oct 5, 2015 at 4:57 AM, Alessandro Benedetti > <abenede...@apache.org> wrote: > > I was doing some studies and analysis, just wondering in your opinion > which > > one is the best approach to use to index in Solr to reach the best > > throughput possible. > > I know that a lot of factor are affecting Indexing time, so let's only > > focus in the feeding approach. > > Let's isolate different scenarios : > > > > *Single Solr Infrastructure* > > > > 1) Xml/Json batch request to /update IndexHandler (xml/json) > > > > 2) SolrJ ConcurrentUpdateSolrClient ( javabin) > > I was thinking this to be the fastest approach for a multi threaded > > indexing application. > > Posting batch of docs if possible per request. > > > > *Solr Cloud* > > > > 1) Xml/Json batch request to /update IndexHandler(xml/json) > > > > 2) SolrJ ConcurrentUpdateSolrClient ( javabin) > > > > 3) CloudSolrClient ( javabin) > > it seems the best approach accordingly to this improvements [1] > > > > What are your opinions ? > > > > A bonus observation should be for using some Map/Reduce big data indexer, > > but let's assume we don't have a big cluster of cpus, but the average > > Indexer server. > > > > > > [1] > > > https://lucidworks.com/blog/indexing-performance-solr-5-2-now-twice-fast/ > > > > > > Cheers > > > > > > -- > > -------------------------- > > > > Benedetti Alessandro > > Visiting card : http://about.me/alessandro_benedetti > > > > "Tyger, tyger burning bright > > In the forests of the night, > > What immortal hand or eye > > Could frame thy fearful symmetry?" > > > > William Blake - Songs of Experience -1794 England > -- -------------------------- Benedetti Alessandro Visiting card - http://about.me/alessandro_benedetti Blog - http://alexbenedetti.blogspot.co.uk "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
