The replication factor is two. I have equally sharded all collections across all nodes. We have a 6 node cluster setup. 300* 6 shards and 2 replicas per shard. I have almost 600 cores per machine
Also one fact is that my zk timeout is in the order of 2-3 minutes. I see zk responses very slow and a lot of outstanding requests (found that out thanks to https://github.com/phunt/) On Thu, Mar 20, 2014 at 2:53 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hours sounds too long indeed. We recently had a client with several > thousand collections, but restart wasn't taking hours... > > Otis > Solr & ElasticSearch Support > http://sematext.com/ > On Mar 20, 2014 5:49 PM, "Erick Erickson" <erickerick...@gmail.com> wrote: > > > How many total replicas are we talking here? > > As in how many shards and, for each shard, > > how many replicas? I'm not asking for a long list > > here, just if you have a bazillion replicas in aggregate. > > > > Hours is surprising. > > > > Best, > > Erick > > > > On Thu, Mar 20, 2014 at 2:17 PM, Chris W <chris1980....@gmail.com> > wrote: > > > Thanks, Shalin. Making clusterstate.json on a collection basis sounds > > > awesome. > > > > > > I am not having problems with #2 . #3 is a major time hog in my > > > environment. I have over 300 +collections and restarting the entire > > cluster > > > takes in the order of hours. (2-3 hour). Can you explain more about > the > > > leaderVoteWait setting? > > > > > > > > > > > > > > > On Thu, Mar 20, 2014 at 1:28 PM, Shalin Shekhar Mangar < > > > shalinman...@gmail.com> wrote: > > > > > >> There are no arbitrary limits on the number of collections but yes > > >> there are practical limits. For example, the cluster state can become > > >> a bottleneck. There is a lot of work happening on finding and > > >> addressing these problems. See > > >> https://issues.apache.org/jira/browse/SOLR-5381 > > >> > > >> Boot up time is because of: > > >> 1) Core discovery, schema/config parsing etc > > >> 2) Transaction log replay on startup > > >> 3) Wait time for enough replicas to become available before leader > > >> election happens > > >> > > >> You can't do much about 1 right now I think. For #2, you can keep your > > >> transaction logs smaller by a hard commit before shutdown. For #3 > > >> there is a leaderVoteWait settings but I'd rather not touch that > > >> unless it becomes a problem. > > >> > > >> On Fri, Mar 21, 2014 at 1:39 AM, Chris W <chris1980....@gmail.com> > > wrote: > > >> > Hi there > > >> > > > >> > Is there a limit on the # of collections solrcloud can support? Can > > >> > zk/solrcloud handle 1000s of collections? > > >> > > > >> > Also i see that the bootup time of solrcloud increases with increase > > in # > > >> > of cores. I do not have any expensive warm up queries. How do i > > speedup > > >> > solr startup? > > >> > > > >> > -- > > >> > Best > > >> > -- > > >> > C > > >> > > >> > > >> > > >> -- > > >> Regards, > > >> Shalin Shekhar Mangar. > > >> > > > > > > > > > > > > -- > > > Best > > > -- > > > C > > > -- Best -- C
