On Sep 26, 2011, at 11:42 AM, Jamie Johnson wrote: > Is there any limitation, be it technical or for sanity reasons, on the > number of shards that can be part of a solr cloud implementation?
The loggly guys ended up hitting a limit somewhere. Essentially, whenever the cloud state is updated, info is read about each shard to update the state (from zookeeper). There is a TODO that I put in there that says something like, "consider updating this incrementally" - usually the data on most shards has not changed, so no reason to read it all. They implemented that today in their own code, but we have not yet done this in trunk. What that places the upper limit at, I don't know - I imagine it takes quite a few shards before it ends up being too much of a problem - they shard by user I believe, so lot's of shards. - Mark Miller lucidimagination.com 2011.lucene-eurocon.org | Oct 17-20 | Barcelona
