Re: Updating 100 documents in one request

2017-03-01 Thread Rick Leir
JSONL is a very simple storage format, and that makes it attractive. You just append lines to a file, with each line being the JSON representation of a document. Much simpler than NoSQL databases. See jsonlines.org http://jsonlines.org/ Cheers -- Rick On March 1, 2017 2:11:49 PM EST, kshitij ty

Re: Updating 100 documents in one request

2017-03-01 Thread kshitij tyagi
thanks everyone for your inputs, we are using solr 5.1 as of now. @rick/walter Can you please explain or provide link for entire set of loaded documents is saved as JSONL in S3 would be helpfull. Regards, Kshitij On Wed, Mar 1, 2017 at 10:06 PM, Walter Underwood wrote: > That is exactly what w

Re: Updating 100 documents in one request

2017-03-01 Thread Walter Underwood
That is exactly what we do. The entire set of loaded documents is saved as JSONL in S3. Very handy for loading up a prod index in test for diagnosis or benchmarking. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 1, 2017, at 8:14 AM, Rick Leir

Re: Updating 100 documents in one request

2017-03-01 Thread Rick Leir
And perhaps put the crawl results in JSONL, so when you get a 404 you can use yesterdays document in a pinch. Cheers -- Rick On March 1, 2017 10:20:21 AM EST, Walter Underwood wrote: >Since I always need to know which document was bad, I back off to >batches of one document when there is a fail

Re: Updating 100 documents in one request

2017-03-01 Thread Walter Underwood
Since I always need to know which document was bad, I back off to batches of one document when there is a failure. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Mar 1, 2017, at 6:25 AM, Erick Erickson wrote: > > What version of Solr? This was a

Re: Updating 100 documents in one request

2017-03-01 Thread Erick Erickson
What version of Solr? This was a pretty long-standing issue that was fixed in Solr 6.1, see: https://issues.apache.org/jira/browse/SOLR-445 Otherwise you really have to write your code to re-transmit sub-packets, perhaps even one at a time when a packet fails. Best, Erick On Wed, Mar 1, 2017 at 3

Updating 100 documents in one request

2017-03-01 Thread kshitij tyagi
Hi Team, I am facing an issue when I am updating more than 1 document on solr. 1. If any 1 document gives 400 error them my other documents are also not updated. How can I approach to solve this? I need my other documents to be indexed which are not giving 400 error. Help appreciated! Regards,