Hi, We are using Solr where there are many update operations. This may not be the right use case for Solr but it's an old application and at this moment we are in no mood to replace Solr with something else.
For one of our use case, we had to use optimistic concurrency for handling concurrent updates. The implementation involves fetching a set of documents, checking a field's value, then constructing the update instruction with an appropriate version for optimistic concurrency, etc. For reducing HTTP communication and network calls I have written a custom update processor. We now push the request document blindly. The request processor modifies the request document based on the documents already present in the index e.g. if field1 of indexedDoc == 'x' set field2 of reqDoc to 'y' index reqDoc For testing, I'm processing 6k unique records (uniqueKey) with roughly 40k updates on the same records in a random sequence. Ideally, at the end state in Solr should be the same after every round of testing -- that's how the algorithm is designed. The client is written in Go. It works perfectly when I run less than four goroutines (concurrent Solr requests). The end state of Solr is exactly the same after every run. But, if I run more I see discrepancies with up to four records out of 6k. The process finishes in 7 to 12 seconds. If I use softCommit=true while pushing the updates to Solr the number of discrepancies are fewer than I think this the case of a race condition where when my update processor is processing a request document it fetches the indexed or tlogged document, another instance of the update processor due to concurrent request is already modified the indexed document. Does my understanding seem correct? If yes, is there any way this can be achieved without using optimistic concurrency in the update processor? Basically, can I somehow serialize the update requests in the update processor? That's a lot of information and also confusing, I guess. Please let me know if any specific details or clarification is required. Thank you. Sachin
