On 8/12/2013 4:50 PM, Thierry Thelliez wrote:
Hello, I am trying to set a four shard system for the first time. I do
not understand why all the shards data are growing at about the same rate
when I push the documents to only one shard.
The four shards represent four calendar years. And for now, on a
development machine, these four shards run on four different ports.
The first shard is started with Zookeeper.
The log of the other shards is filed with something like:
7882051 [qtp1154079020-1245] INFO
org.apache.solr.update.processor.LogUpdateProcessor – [collection1]
webapp=/solr path=/update params={distrib.from=
http://x.y.z.4:50121/solr/collection1/&update.distrib=TOLEADER&wt=javabin&version=2}
{add=[14939-96467-304 (1443204912169091072), 14939-96467-308
(1443204912179576832), 14939-96467-310 (1443204912185868288),
14939-96467-311 (1443204912192159744), 14939-96467-313
(1443204912204742656), 14939-96467-314 (1443204912220471296),
14939-96467-318 (1443204912239345664), 14939-96467-319
(1443204912250880000), 14939-96467-322 (1443204912257171456),
14939-96467-324 (1443204912263462912)]} 0 282
What is getting written to the other shards? Is a separate index computed
on all four shards? I thought that when pushing a document to one shard,
only that shard would update its index.
There are two possibilities.
1) You don't have four shards, you have four replicas of one shard. If
this is happening, then they all will receive all documents.
2) You are using a router like compositeId instead of implicit. This
will calculate the hash of the id field and evenly divide the documents
among all the shards in the collection according to the hash value. If
you create the collection with the implicit router, then documents
should be indexed by the shard that received them.
To see what router you have, click on Cloud in the admin UI, then click
on Tree. Click the arrow to the left of '/collections' to open it.
Click on collection1 (or whichever you are actually using) -- the actual
name, not the arrow. Underneath the table that appears to the right
will be "router" and its value.
Thanks,
Shawn
