Re: Risk of Over Sharding

2015-09-23 Thread Alessandro Benedetti
Thanks Erick, your insights are really useful. Honestly I agree and when the prototyping will come I will definitely proceed like you suggested ! Cheers 2015-09-23 16:53 GMT+01:00 Erick Erickson : > Sure, prototyping is the best answer ;). > > Well, the biggest question is "how long > does each

Re: Risk of Over Sharding

2015-09-23 Thread Erick Erickson
Sure, prototyping is the best answer ;). Well, the biggest question is "how long does each shard spend on the query"? Adding more shards will likely decrease the time each shard takes to process the first-pass query. But if your base is 50ms, and sharding some more takes it to 40ms I don't think

Risk of Over Sharding

2015-09-23 Thread Alessandro Benedetti
It's quite common to hear about the benefit of sharding. Until we reach the I/O bound on our machines, sharding is likely to reduce the query time. Furthermore working on smaller indexes will make the single searches faster on the smaller nodes. But what about the other way around ? What if we actu