luceneMatchVersion is something that cause differences. You may want to
try 6.3 version to be on same page as Amrit's code.
On Fri, Jul 21, 2017 at 6:08 PM, Hendrik Haddorp <hendrik.hadd...@gmx.net>
wrote:
> Thanks for trying to reproduce my issue.
>
> I'm using a Solr Cloud, my collection was quite small, only a 50-500
> documents, with one shard and a replication factor of 3. I updated all of
> the documents in one request. Beside that the flow is pretty much like
> yours.
>
> The goal of my code was to add a field to all documents that did not
> contain the field already, which should actually have been every document
> in the collection. So the code does a query and gets a few fields for each
> document. Then I use an atomic update to add two fields. In the end I send
> all fields to Solr in one request. The whole thing did then loop over a few
> hundred collections. In most cases it worked just fine but in some it
> failed. If so I could reproduce the issue on that collection constantly. In
> an earlier version I accidentally tried to add a field instead of setting
> it and that had caused an exception as the field only allowed a single
> value. At least in one case the version conflict later on happened on a
> collection that had this issue before. don't really think that causes it so
> just stating for the sake of completeness. I was not using a document
> centric version field but the normal _version_ field.
>
> Even though I'm running on Solr 6.3 my config still contains
> "<luceneMatchVersion>6.2.1</luceneMatchVersion>". Not sure if that might
> have an effect.
>
> The main code is this:
> SolrQuery query = new SolrQuery();
> query.setQuery(documentsMatchingQueryQ);
> query.setRows(Integer.MAX_VALUE);
> query.setFields(documentsMatchingQueryFL);
>
> QueryResponse queryResponse =
> solr.query(collectionName, query);
> if (queryResponse.getStatus() != 0) {
> LOGGER.error("request failed,
> status: ", queryResponse);
> throw new
> IllegalStateException("request failed");
> }
> LOGGER.info("{} | documents to
> process: {}", collectionName, queryResponse.getResults().getNumFound());
> if
> (queryResponse.getResults().getNumFound()
> > 0) {
> LOGGER.trace("{} | start
> processing", collectionName);
> List<SolrInputDocument> docs = queryResponse.getResults().stream()
> .map(documentGenerator::apply)
> .peek(doc ->
> LOGGER.trace("{} doc: {}", collectionName, doc))
> .collect(Collectors.toList());
> if (!reportOnly) {
> LOGGER.trace("{} | updating
> solr", collectionName);
> solr.add(collectionName, docs);
> solr.commit(collectionName, false, false, true);
>
> }
> LOGGER.trace("{} | processing
> done", collectionName);
> }
>
> The documentGenerator looks like this:
> originalDoc -> {
> SolrInputDocument doc = new SolrInputDocument();
>
> doc.addField("id", originalDoc.getFieldValue("id"));
> doc.addField("__docId__",
> Collections.singletonMap("set", originalDoc.getFieldValue("classification")
> + ":" + originalDoc.getFieldValue("id")));
> doc.addField("systemModified",
> Collections.singletonMap("set", originalDoc.getFieldValue("las
> tModified")));
> doc.addField("_version_",
> originalDoc.getFieldValue("_version_"));
>
> return doc;
>
> }
>
> On 21.07.2017 22:33, Amrit Sarkar wrote:
>
>> Hendrik,
>>
>> Ran a little test on 6.3, with infinite atomic updates with optimistic
>> concurrency,
>> cannot *reproduce*:
>>
>> List<SolrInputDocument> docs = new ArrayList<>();
>>
>>> SolrInputDocument document = new SolrInputDocument();
>>> document.addField("id", String.valueOf(1));
>>> document.addField("external_version_field_s",
>>> System.currentTimeMillis()); // normal update
>>> docs.add(document);
>>> UpdateRequest updateRequest = new UpdateRequest();
>>> updateRequest.add(docs);
>>> client.request(updateRequest, collection);
>>> updateRequest = new UpdateRequest();
>>> updateRequest.commit(client, collection);
>>>
>>> while (true) {
>>> QueryResponse response = client.query(new
>>> ModifiableSolrParams().add("q", "id:1"));
>>> System.out.println(response.getResults().get(0).get("_version_"));
>>> docs = new ArrayList<>();
>>> document = new SolrInputDocument();
>>> document.addField("id", String.valueOf(1));
>>> Map<String, String> map = new HashMap<>();
>>> map.put("set", createSentance(1)); // atomic map value
>>> document.addField("external_version_field_s", map);
>>> document.addField("_version_", response.getResults().get(0).g
>>> et("_version_"));
>>> docs.add(document);
>>> updateRequest = new UpdateRequest();
>>> updateRequest.add(docs);
>>> client.request(updateRequest, collection);
>>> updateRequest = new UpdateRequest();
>>> updateRequest.commit(client, collection);
>>> }
>>>
>>> Maybe you can let us know more details how the update been made?
>>>
>> Amrit Sarkar
>> Search Engineer
>> Lucidworks, Inc.
>> 415-589-9269
>> www.lucidworks.com
>> Twitter http://twitter.com/lucidworks
>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>>
>> On Fri, Jul 21, 2017 at 10:36 PM, Hendrik Haddorp <
>> hendrik.hadd...@gmx.net>
>> wrote:
>>
>> Hi,
>>>
>>> I can't find anything about this in the Solr logs. On the caller side I
>>> have this:
>>> Error from server at http://xxxxx_shard1_replica2: version conflict for
>>> xxxxx expected=1573538179623944192 actual=1573546159565176832
>>> org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error
>>> from server at http://xxxxx_shard1_replica2: version conflict for xxxxx
>>> expected=1573538179623944192 actual=1573546159565176832
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpda
>>> te(CloudSolrClient.java:765)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.sendReques
>>> t(CloudSolrClient.java:1173)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWit
>>> hRetryOnStaleState(CloudSolrClient.java:1062)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.request(Cl
>>> oudSolrClient.java:1004)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest
>>> .java:149)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> ...
>>> Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrE
>>> xception:
>>> Error from server at http://xxxxx_shard1_replica2: version conflict for
>>> xxxxx expected=1573538179623944192 actual=1573546159565176832
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMeth
>>> od(HttpSolrClient.java:593)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>>> pSolrClient.java:262)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>>> pSolrClient.java:251)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest
>>> (LBHttpSolrClient.java:435)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(L
>>> BHttpSolrClient.java:387)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$dir
>>> ectUpdate$0(CloudSolrClient.java:742) ~[solr-solrj-6.3.0.jar:6.3.0
>>> a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:52:43]
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> ~[?:1.8.0_131]
>>> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
>>> xecutor.lambda$execute$0(ExecutorUtil.java:229)
>>> ~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
>>> shalin - 2016-11-02 19:52:43]
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1142)
>>> ~[?:1.8.0_131]
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:617)
>>> ~[?:1.8.0_131]
>>> at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
>>>
>>> The version "1573546159565176832" does not exist. It looks a bit like the
>>> update was first creating a new value and then checks against it.
>>>
>>> regards,
>>> Hendrik
>>>
>>>
>>> On 21.07.2017 18:21, Amrit Sarkar wrote:
>>>
>>> Hendrik,
>>>>
>>>> Can you list down the error snippet so that we can refer the code where
>>>> exactly that is happening.
>>>>
>>>>
>>>> Amrit Sarkar
>>>> Search Engineer
>>>> Lucidworks, Inc.
>>>> 415-589-9269
>>>> www.lucidworks.com
>>>> Twitter http://twitter.com/lucidworks
>>>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>>>>
>>>> On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <
>>>> hendrik.hadd...@gmx.net
>>>> wrote:
>>>>
>>>> Hi,
>>>>
>>>>> when I try to use an atomic update in conjunction with optimistic
>>>>> concurrency Solr sometimes complains that the version I passed in does
>>>>> not
>>>>> match. The version in my request however match to what is stored and
>>>>> what
>>>>> the exception states as the actual version does not exist in the
>>>>> collection
>>>>> at all. Strangely this does only happen sometimes but once it happens
>>>>> for a
>>>>> collection it seems to stay like that. Any idea why that might happen?
>>>>>
>>>>> I'm using Solr 6.3 in Cloud mode with SolrJ.
>>>>>
>>>>> regards,
>>>>> Hendrik
>>>>>
>>>>>
>>>>>
>
