This is my clusterstate.json: https://gist.github.com/bretthoerner/0098f741f48f9bb51433
And these are my core sizes (note large ones are sorted to the end): https://gist.github.com/bretthoerner/f5b5e099212194b5dff6 I've only "heavily sent" 2 shards by now (I'm sharding by hour and it's been running for 2). There *is* a little old data in my stream, but not that much (like <5%). What's confusing to me is that 5 of them are rather large, when I'd expect 2 of them to be. On Tue, Oct 8, 2013 at 5:45 PM, Yonik Seeley <ysee...@gmail.com> wrote: > On Tue, Oct 8, 2013 at 6:29 PM, Brett Hoerner <br...@bretthoerner.com> > wrote: > > I'm curious what the later "shard-local" bits do, if anything? > > > > I have a very large cluster (256 shards) and I'm sending most of my data > > with a single "composite", e.g. 1234!<unique_id>, but I'm noticing the > data > > is being split among many of the shards. > > That shouldn't be the case. All of your shards should have a lower > hash value with all 0 bits and an upper hash value of all 1s (i.e. > 0x????0000 to 0x????ffff) > So you see any shards where that's not true? > > Also, is the router set to compositeId? > > -Yonik > > > My guess right now is that since I'm only using the default 16 bits my > data > > is being split across multiple shards (because of my high # of shards). > > > > Thanks, > > Brett >
