Re: Performance potential for updating (reindexing) documents

2016-04-02 Thread Shawn Heisey
On 4/1/2016 8:56 PM, Erick Erickson wrote: > bq: The bottleneck is definitely Solr. > > Since you commented out the server.add(doclist), you're right to focus > there. I've seen > a few things that help. > > 1> batch the documents, i.e. in the doclist above the list should be > on the order of 1,00

Re: Performance potential for updating (reindexing) documents

2016-04-01 Thread Erick Erickson
Shawn: bq: The bottleneck is definitely Solr. Since you commented out the server.add(doclist), you're right to focus there. I've seen a few things that help. 1> batch the documents, i.e. in the doclist above the list should be on the order of 1,000 docs. Here are some numbers I worked up one tim

Re: Performance potential for updating (reindexing) documents

2016-03-31 Thread Shawn Heisey
On 3/24/2016 11:57 AM, tedsolr wrote: > My post was scant on details. The numbers I gave for collection sizes are > projections for the future. I am in the midst of an upgrade that will be > completed within a few weeks. My concern is that I may not be able to > produce the throughput necessary to

Re: Performance potential for updating (reindexing) documents

2016-03-24 Thread Erick Erickson
ses), and have just begun the process of tuning > in a test environment. > > After some more weeks of testing and tweaking I hope to get to 5000 > updates/sec, but even that may not be enough. So my main concern is that > this business model (of updating entire collections about once a

Re: Performance potential for updating (reindexing) documents

2016-03-24 Thread tedsolr
://lucene.472066.n3.nabble.com/Performance-potential-for-updating-reindexing-documents-tp4265861p4265922.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Performance potential for updating (reindexing) documents

2016-03-24 Thread Erick Erickson
docs/sec? > > Using SolrJ for querying and indexing against a v5.2.1 cloud. > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Performance-potential-for-updating-reindexing-documents-tp4265861.html > Sent from the Solr - User mailing list archive at Nabble.com.

Performance potential for updating (reindexing) documents

2016-03-24 Thread tedsolr
y small in size. What's the best possible throughput I can expect? 1000 docs/sec? 5000 docs/sec? Using SolrJ for querying and indexing against a v5.2.1 cloud. -- View this message in context: http://lucene.472066.n3.nabble.com/Performance-potential-for-updating-reindexing-documents-tp426586