On 10/12/2014 12:26 PM, vidit.asthana wrote: > I have a strange problem where select q=*:* is returning different number of > documents. Sometime its returning numFound = 5866712 and sometimes it > returns numFound = 5852274. *numFound is always one of these 2 values.* > > Here is the query: > > *http://localhost:5011/solr/mycollection/select?q=*:*&rows=0* > > > I am running Solr in cloud mode and this problem is occurring with both > solr-4.5.1 and solr-4.10.0. I have exactly same data indexed in both > versions. 4.5.1 is running on a 8 nodes cluster (4x2 shards) and solr-4.10.0 > is running on a 4 node (2x2 shards)cluster.
I really need to make a wiki page for this. It would save so much typing! I also need to boil it down to a small-scale real-world example and show how the numbers get calculated and what goes wrong, which means I need to have a complete understanding of the problem, and at this moment, I don't have that. This is a problem that's unique to distributed indexes. What causes it is having documents with the same value in the uniqueKey field indexed in more than one shard. It is not a bug, it's a result of the way that results from multiple shards are combined into one result. The only way to "fix" this problem would involve so much additional processing that it would make all queries extremely slow. If you're using automatic document routing, then your routing algorithm may have changed at some point, and you didn't re-index. If you're using manual document routing, then some documents were indexed on the wrong shard, and later indexed on another shard as well. Preventing the problem is easy -- always index documents onto the correct shard. Fixing the problem at this point might involve clearing your index and re-indexing from scratch, unless you can figure out which documents have been indexed on more than one shard and you can delete them from the incorrect shard(s). Thanks, Shawn
