I'm afraid I'm note completely clear about your scenario. Let me say how I understand what you're saying, and what I've done in the past.
Firstly, I take it you are using Solr 3.x (from your reference to a 'shards' parameter. Secondly, you refer to a 'stripe' as one set of nodes, one for each shard, that are enough to allow querying your whole collection. Having created the concept of a 'slice', you then hardwire the 'shards' parameter in solrconfig.xml in each machine in that slice to point to all the other nodes in that same slice. Then you point your load balancer at some boxes, which will do distributed queries. Now, by the sounds of it, every box on your setup could do this, they all have a shards parameter set up. Minimally, you'll want at least one box from each slice, otherwise you'll have slices that aren't receiving queries. But could you include all of your boxes, and have all of them handling the query distribution work? I guess you could, but I'd suggest another architecture. In the setup you describe, if you loose one node, you loose an entire slice. However, if a distributed query comes into another node in the slice, the load balancer may well not notice (unless you make the healthcheck itself do a distributed search) and things could get messy. What I've done is set up a VIP in my load balancer for each and every node that can service a shard. Repeat that for each shard that I have. Let's say I have four shards, I'll end up with four VIPs. I then put those four VIPs into my shards parameter in solrconfig.xml on all of my hosts, regardless of what shard/slice. Then, I create another VIP that includes all of my nodes in it. That is the one that I hand to my application. This way, you can loose any node in any shard and the thing should keep on going. Obviously I'm talking about slaves here. There will be a master for each shard which each of these nodes pull their indexes from. Hope this is helpful. Upayavira On Tue, Jan 29, 2013, at 09:35 PM, Lee, Peter wrote: > I would appreciate people's experience on the following load balancing > question... > > We currently have solr configured in shards across multiple machines to > handle our load. That is, a request being sent to any one of these > servers will cause that server to query the rest of the servers in that > "stripe" (we use the term "stripe" to refer to a set of servers that > point to each other with the shard parameter). > > We currently have all servers in a stripe registered with our load > balancer. Thus, requests are being spread out across all servers in the > stripe...but of course requests to any shard generates additional traffic > on all shards in that stripe. > > My question (finally) is this: Has anyone determined if it is better to > place only a few (that is, not all) of the shards in a stripe on the load > balancer as versus ALL of the shards in a stripe on the load balancer? It > seemed to me at first that it would not make much of a difference, but > then I realized that this would really depend on the relative costs of a > few different steps (one step would be the cost of collecting all of the > responses from the other servers in the shard to formulate the final > answer. Another step would be the cost of generating more traffic between > the shards, etc.). > > So what I am trying to ask is this: If we had 6 servers in a "stripe" (6 > servers set up as shards to support a single query), would there be any > advantage with respect to handling load to only place ONE or TWO of the > shards on the load balancer as versus putting ALL shards on the load > balancer? > > We can test this empirically but if the community already has gotten a > feel for the best practice in this situation I would be happy to learn > from your experience. I could not find anything online that spoke to this > particular situation. > > Thanks. > > Peter S. Lee > Senior Software Engineer > ProQuest > 789 E. Eisenhower Parkway > Ann Arbor, MI, 48106-1346 > USA > 734-761-4700 x72025 > peter....@proquest.com > www.proquest.com > > ProQuest...Start here > InformationWeek 500 Top > Innovator<http://www.proquest.com/en-US/aboutus/pressroom/09/20090922.shtml> >
