Re: Replicas in Recovery During Atomic Updates

2020-08-19 Thread Anshuman Singh
Hi, Anyone has any idea about this issue? Apart from the errors in the previous email, facing below errors frequently: 2020-08-19 11:56:09.467 ERROR (qtp1546693040-32) [c:collection_4 s:shard3 r:core_node13 x:collection_4_shard3_replica_n10] o.a.s.u.SolrCmdDistributor java.io.IOException: Request

Re: Replicas in Recovery During Atomic Updates

2020-08-10 Thread Anshuman Singh
Just to give you an idea, this is how we are ingesting: {"id": 1, "field1": {"inc": 20}, "field2": {"inc": 30}, "field3": 40. "field4": "some string"} We are using Solr-8.5.1. We have not configured any update processor. Hard commit happens every minute or at 100k docs, soft commit happens every

Re: Replicas in Recovery During Atomic Updates

2020-08-10 Thread Jörn Franke
How do you ingest it exactly with Atomtic updates ? Is there an update processor in-between? What are your settings for hard/soft commit? For the shared going to recovery - do you have a log entry or something ? What is the Solr version? How do you setup ZK? > Am 10.08.2020 um 16:24 schrieb

Re: Replicas in Recovery During Atomic Updates

2020-08-10 Thread Erick Erickson
Good question, what do the logs say? You’ve provided very little information to help diagnose the issue. As to your observation that atomic updates are expensive, that’s true. Under the covers, Solr has to go out and fetch the document, overlay your changes and then re-index the full document. So,

Replicas in Recovery During Atomic Updates

2020-08-10 Thread Anshuman Singh
Hi, We have a SolrCloud cluster with 10 nodes. We have 6B records ingested in the Collection. Our use case requires atomic updates ("inc") on 5 fields. Now almost 90% documents are atomic updates and as soon as we start our ingestion pipelines, multiple shards start going into recovery, sometimes