Re: indexing error - 6.3.0

2017-01-19 Thread Joe Obernberger
Another data point - the 5 node cluster does have another collection on it that is large (maybe 500G in HDFS) that did have field guessing enabled on it, but it is a static collection (I'm not adding data to it). I've just removed that collection and am running the test again - it's gotten a l

Re: indexing error - 6.3.0

2017-01-19 Thread Joe Obernberger
Thank you Erick! For this scenario, I was defining the schema manually (editing managed_schema and pushing to zookeeper), but didn't realize that I had left the field guessing block in the solrconfig.xml file enabled. I've now disabled the field guessing, but still getting errors when indexin

Re: indexing error - 6.3.0

2017-01-19 Thread Erick Erickson
It looks to me like you're using "field guessing". For production systems I generally don't recommend this. The problem is that it makes the best estimate that it can based on the first document for any given field. So it sees a field with the value 1 and tries to make the field an int. Then 100 do

Re: indexing error - 6.3.0

2017-01-18 Thread Joe Obernberger
Hi All - I've been trying to debug this, but it keeps occurring. Even if I do 100 at a time, or 50 at a time, eventually I get the below stack trace. I've also adjusted the autoSoftCommit and autoCommit times to a variety of values. It stills fails after a time; typically around 27-50 million