Re: Architecture suggestions

2017-03-24 Thread Shawn Heisey
On 3/24/2017 7:47 AM, vrindavda wrote: > In my case query rate will be average or say low, 100-120 concorrent > requests. That is not a low query rate. A low query rate would be X queries per second, where X is a small single-digit number. If there are 100 *simultaneous* requests, then the quer

Re: Architecture suggestions

2017-03-24 Thread vrindavda
Thanks Shawn, In my case query rate will be average or say low, 100-120 concorrent requests. As per my understanding replica too aid shards in getting result documents, correct if I am wrong. Moreover, I intend to have fault tolerant architecture, hence opting for shards/replicas on different se

Re: Architecture suggestions

2017-03-24 Thread Shawn Heisey
On 3/24/2017 1:15 AM, vrindavda wrote: > Thanks Erick and Emir , for your prompt reply. > > We are expecting around 50M documents to sit on 80GB . I understand that > there is no equation to predict the number/size of server. But considering > to have minimal fault tolerant architecture, Will 2 sha

Re: Architecture suggestions

2017-03-24 Thread vrindavda
Thanks Erick and Emir , for your prompt reply. We are expecting around 50M documents to sit on 80GB . I understand that there is no equation to predict the number/size of server. But considering to have minimal fault tolerant architecture, Will 2 shards and 2 replicas with 128GB RAM, 4 core solr i

Re: Architecture suggestions

2017-03-23 Thread David Hastings
Yeah coming up with a "perfect" machine for your use is completely trial and error. for me personally i found that on one machine with 24 cores, 148gb ram, handles one solr instance with 4 cores, a 16mil records sitting at 400gb, a 53mil records sitting at 160gb, and a 108mil records sitting at 3

Re: Architecture suggestions

2017-03-23 Thread Erick Erickson
I've seen single nodes handle 10M docs using 64G of heap (using Zing). I've seen 300M in 12G of memory. There's absolutely no way to tell. See: https://lucidworks.com/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/ for a methodology to answer the question with _yo

Re: Architecture suggestions

2017-03-23 Thread Emir Arnautovic
Hi Vrindavda, It is hard to tell anything without testing and details on what/how is indexed, how it is going to be queried and what are latency/throughput requirements. 25M or 12.5M documents per shard might be too much if you have strict latency requirements, but testing is the only way to