FYI. We finally tracked down the problem.... at least 99.9% sure at this point, and it was staring me in the face the whole time - just never noticed:

[{"id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1","channel": {"add": "preet"},"channel": 
{"add": "adam"}}]

Look at the JSON... It's trying to add two "channel" array elements... Should 
have been:

[{"id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1","channel": {"add": "preet"}},
 {"id":"4b2c4d09-31e2-4fe2-b767-3868efbdcda1","channel": {"add": "adam"}}]

I half wonder how it chose to interpret that particular chunk of json, but either way, I think the origin of our issue is resolved.


From what I'm reading on JSON - this isn't valid syntax at all. I'm guessing that SOLR doesn't actually validate the JSON, and it's parser is just creating something weird in that situation like a new request for a whole new document.

-- Nathan


On 07/15/2014 07:19 PM, Nathan Neulinger wrote:
Issue was closed in Jira requesting it be discussed here first. Looking for any 
diagnostic assistance on this issue with
4.8.0 since it is intermittent and occurs without warning.

Setup is two nodes, with external zk ensemble. Nodes are accessed round-robin 
on EC2 behind an ELB.

Schema has:

<schema name="hive" version="1.5">
...
    <field name="timestamp" type="long" indexed="false" stored="true" required="true" 
multiValued="false"
omitNorms="true" />
...


Most of the updates are working without issue, but randomly we'll get the above 
failure, even though searches before and
after the update clearly indicate that the document had the timestamp field in 
it. The error occurs when the second node
does it's distrib operation against the first node.

Diagnostic details are all in the jira issue. Can provide more as needed, but 
would appreciate any suggestions on what
to try or to help diagnose this other than just trying to throw thousands of 
requests at it in round-robin between the
two instances to see if it's possible to reproduce the issue.

-- Nathan

------------------------------------------------------------
Nathan Neulinger                       nn...@neulinger.org
Neulinger Consulting                   (573) 612-1412

--
------------------------------------------------------------
Nathan Neulinger                       nn...@neulinger.org
Neulinger Consulting                   (573) 612-1412

Reply via email to