Router is definitely compositeId. To be clear, data isn't being spread evenly... it's like it's *almost* working. It's just odd to me that I'm slamming in data that's 99% of one _route_ key yet after a few minutes (from a fresh empty index) I have 2 shards with a sizeable amount of data (68M and 128M) and the rest are very small as expected.
The fact that two are receiving so much makes me think my data is being split into two shards. I'm trying to debug more now. On Tue, Oct 8, 2013 at 5:45 PM, Yonik Seeley <ysee...@gmail.com> wrote: > On Tue, Oct 8, 2013 at 6:29 PM, Brett Hoerner <br...@bretthoerner.com> > wrote: > > I'm curious what the later "shard-local" bits do, if anything? > > > > I have a very large cluster (256 shards) and I'm sending most of my data > > with a single "composite", e.g. 1234!<unique_id>, but I'm noticing the > data > > is being split among many of the shards. > > That shouldn't be the case. All of your shards should have a lower > hash value with all 0 bits and an upper hash value of all 1s (i.e. > 0x????0000 to 0x????ffff) > So you see any shards where that's not true? > > Also, is the router set to compositeId? > > -Yonik > > > My guess right now is that since I'm only using the default 16 bits my > data > > is being split across multiple shards (because of my high # of shards). > > > > Thanks, > > Brett >
