Thanks for trying to reproduce my issue.
I'm using a Solr Cloud, my collection was quite small, only a 50-500
documents, with one shard and a replication factor of 3. I updated all
of the documents in one request. Beside that the flow is pretty much
like yours.
The goal of my code was to add a field to all documents that did not
contain the field already, which should actually have been every
document in the collection. So the code does a query and gets a few
fields for each document. Then I use an atomic update to add two fields.
In the end I send all fields to Solr in one request. The whole thing did
then loop over a few hundred collections. In most cases it worked just
fine but in some it failed. If so I could reproduce the issue on that
collection constantly. In an earlier version I accidentally tried to add
a field instead of setting it and that had caused an exception as the
field only allowed a single value. At least in one case the version
conflict later on happened on a collection that had this issue before.
don't really think that causes it so just stating for the sake of
completeness. I was not using a document centric version field but the
normal _version_ field.
Even though I'm running on Solr 6.3 my config still contains
"<luceneMatchVersion>6.2.1</luceneMatchVersion>". Not sure if that might
have an effect.
The main code is this:
SolrQuery query = new SolrQuery();
query.setQuery(documentsMatchingQueryQ);
query.setRows(Integer.MAX_VALUE);
query.setFields(documentsMatchingQueryFL);
QueryResponse queryResponse =
solr.query(collectionName, query);
if (queryResponse.getStatus() != 0) {
LOGGER.error("request failed,
status: ", queryResponse);
throw new
IllegalStateException("request failed");
}
LOGGER.info("{} | documents to
process: {}", collectionName, queryResponse.getResults().getNumFound());
if
(queryResponse.getResults().getNumFound() > 0) {
LOGGER.trace("{} | start
processing", collectionName);
List<SolrInputDocument> docs = queryResponse.getResults().stream()
.map(documentGenerator::apply)
.peek(doc ->
LOGGER.trace("{} doc: {}", collectionName, doc))
.collect(Collectors.toList());
if (!reportOnly) {
LOGGER.trace("{} | updating
solr", collectionName);
solr.add(collectionName, docs);
solr.commit(collectionName, false, false, true);
}
LOGGER.trace("{} | processing
done", collectionName);
}
The documentGenerator looks like this:
originalDoc -> {
SolrInputDocument doc = new SolrInputDocument();
doc.addField("id", originalDoc.getFieldValue("id"));
doc.addField("__docId__",
Collections.singletonMap("set",
originalDoc.getFieldValue("classification") + ":" +
originalDoc.getFieldValue("id")));
doc.addField("systemModified",
Collections.singletonMap("set", originalDoc.getFieldValue("lastModified")));
doc.addField("_version_",
originalDoc.getFieldValue("_version_"));
return doc;
}
On 21.07.2017 22:33, Amrit Sarkar wrote:
Hendrik,
Ran a little test on 6.3, with infinite atomic updates with optimistic
concurrency,
cannot *reproduce*:
List<SolrInputDocument> docs = new ArrayList<>();
SolrInputDocument document = new SolrInputDocument();
document.addField("id", String.valueOf(1));
document.addField("external_version_field_s", System.currentTimeMillis()); //
normal update
docs.add(document);
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.add(docs);
client.request(updateRequest, collection);
updateRequest = new UpdateRequest();
updateRequest.commit(client, collection);
while (true) {
QueryResponse response = client.query(new ModifiableSolrParams().add("q",
"id:1"));
System.out.println(response.getResults().get(0).get("_version_"));
docs = new ArrayList<>();
document = new SolrInputDocument();
document.addField("id", String.valueOf(1));
Map<String, String> map = new HashMap<>();
map.put("set", createSentance(1)); // atomic map value
document.addField("external_version_field_s", map);
document.addField("_version_",
response.getResults().get(0).get("_version_"));
docs.add(document);
updateRequest = new UpdateRequest();
updateRequest.add(docs);
client.request(updateRequest, collection);
updateRequest = new UpdateRequest();
updateRequest.commit(client, collection);
}
Maybe you can let us know more details how the update been made?
Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2
On Fri, Jul 21, 2017 at 10:36 PM, Hendrik Haddorp <hendrik.hadd...@gmx.net>
wrote:
Hi,
I can't find anything about this in the Solr logs. On the caller side I
have this:
Error from server at http://xxxxx_shard1_replica2: version conflict for
xxxxx expected=1573538179623944192 actual=1573546159565176832
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error
from server at http://xxxxx_shard1_replica2: version conflict for xxxxx
expected=1573538179623944192 actual=1573546159565176832
at
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:765)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1173)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWit
hRetryOnStaleState(CloudSolrClient.java:1062)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
...
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at http://xxxxx_shard1_replica2: version conflict for
xxxxx expected=1573538179623944192 actual=1573546159565176832
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$dir
ectUpdate$0(CloudSolrClient.java:742) ~[solr-solrj-6.3.0.jar:6.3.0
a66a44513ee8191e25b477372094bfa846450316 - shalin - 2016-11-02 19:52:43]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[?:1.8.0_131]
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
xecutor.lambda$execute$0(ExecutorUtil.java:229)
~[solr-solrj-6.3.0.jar:6.3.0 a66a44513ee8191e25b477372094bfa846450316 -
shalin - 2016-11-02 19:52:43]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[?:1.8.0_131]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
The version "1573546159565176832" does not exist. It looks a bit like the
update was first creating a new value and then checks against it.
regards,
Hendrik
On 21.07.2017 18:21, Amrit Sarkar wrote:
Hendrik,
Can you list down the error snippet so that we can refer the code where
exactly that is happening.
Amrit Sarkar
Search Engineer
Lucidworks, Inc.
415-589-9269
www.lucidworks.com
Twitter http://twitter.com/lucidworks
LinkedIn: https://www.linkedin.com/in/sarkaramrit2
On Fri, Jul 21, 2017 at 9:50 PM, Hendrik Haddorp <hendrik.hadd...@gmx.net
wrote:
Hi,
when I try to use an atomic update in conjunction with optimistic
concurrency Solr sometimes complains that the version I passed in does
not
match. The version in my request however match to what is stored and what
the exception states as the actual version does not exist in the
collection
at all. Strangely this does only happen sometimes but once it happens
for a
collection it seems to stay like that. Any idea why that might happen?
I'm using Solr 6.3 in Cloud mode with SolrJ.
regards,
Hendrik
