Re: error reporting during indexing

2015-10-01 Thread Erick Erickson
bq: If there is a problem writing the segment, a permission error, Highly doubtful that this'll occur. When an IndexWriter is opened, the first thing that's (usually) done is write to the lock file to keep other Solr's from writing. That should fail right off the bat, far before any docs are actua

Re: error reporting during indexing

2015-09-29 Thread Alessandro Benedetti
Hi Matteo, at this point I would suggest you this reading by Erick: https://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ If i am not wrong when the document is indexed ( simplifying) : 1) The document is added to the current segment in memory 2) When a so

Re: error reporting during indexing

2015-09-29 Thread Matteo Grolla
Hi Erik, it's a curiosity question. When I add a document it's buffered by Solr and can (apparently is) be parsed to verify it matches the schema. But it's not written to a segment file until a commit is issued. If there is a problem writing the segment, a permission error, isn't this a case wh

Re: error reporting during indexing

2015-09-28 Thread Erick Erickson
You shouldn't be losing errors with HttpSolrServer. Are you seeing evidence that you are or is this mostly a curiosity question? Do not it's better to batch up docs, your throughput will increase a LOT. That said, when you do batch (e.g. send 500 docs per update or whatever) and you get an error b

error reporting during indexing

2015-09-28 Thread Matteo Grolla
Hi, if I need fine grained error reporting I use Http Solr server and send 1 doc per request using the add method. I report errors on exceptions of the add method, I'm using autocommit so I'm not seing errors related to commit. Am I loosing some errors? Is there a better way? Thanks