RE: SolrCloud recommended I/O RAID level

2019-08-04 Thread Kaminski, Adi
We'll take all these into consideration in order to size and simulate both indexing and search flow in load. Thanks again, Adi -Original Message- From: Erick Erickson Sent: Sunday, August 4, 2019 6:51 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level

Re: SolrCloud recommended I/O RAID level

2019-08-04 Thread Erick Erickson
t;> >> · Questions: >> >> o Will additional replicas will assist with QPS ? As we will basically are >> having ratio of 1 vCPU per 1 shard (the mster replica) >> >> o Wouldn’t additional replicas will overclock per total amout of vCPUs, >> an

Re: SolrCloud recommended I/O RAID level

2019-08-04 Thread Shawn Heisey
On 8/4/2019 8:53 AM, Kaminski, Adi wrote: Erick - thanks a lot for answering and sharing the below article, it's very helpful ! I have another follow-up question - assuming we have 400 vCPUs across our SolrCloud cluster nodes, will it be better to have 400 shards with replication factor 2 or

RE: SolrCloud recommended I/O RAID level

2019-08-04 Thread Kaminski, Adi
What utilizes better the CPUs - shards/or their replicas ? Thanks, Adi -Original Message- From: Erick Erickson Sent: Thursday, August 1, 2019 6:48 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level Yes, I’m exactly talking about the idea of someone pu

Re: SolrCloud recommended I/O RAID level

2019-08-01 Thread Erick Erickson
queries and lower QPS ? > > > > Thanks, > > Adi > > > > -Original Message- > From: Erick Erickson > Sent: Thursday, August 1, 2019 2:03 PM > To: solr-user@lucene.apache.org > Subject: Re: SolrCloud recommended I/O RAID level > > >

RE: SolrCloud recommended I/O RAID level

2019-08-01 Thread Kaminski, Adi
k per total amout of vCPUs, and actually cause delays in queries and lower QPS ? Thanks, Adi -Original Message- From: Erick Erickson Sent: Thursday, August 1, 2019 2:03 PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level “why would I need a replic

Re: SolrCloud recommended I/O RAID level

2019-08-01 Thread Erick Erickson
, July 30, 2019 9:44 PM > To: solr-user@lucene.apache.org > Subject: Re: SolrCloud recommended I/O RAID level > > On 7/30/2019 12:12 PM, Kaminski, Adi wrote: >> Indeed RAID10 with both mirroring and striping should satisfy the >> need, but per some benchmarks in the network there is

RE: SolrCloud recommended I/O RAID level

2019-07-31 Thread Kaminski, Adi
PM To: solr-user@lucene.apache.org Subject: Re: SolrCloud recommended I/O RAID level On 7/30/2019 12:12 PM, Kaminski, Adi wrote: > Indeed RAID10 with both mirroring and striping should satisfy the > need, but per some benchmarks in the network there is still an impact > on write performance

Re: SolrCloud recommended I/O RAID level

2019-07-30 Thread Shawn Heisey
On 7/30/2019 12:12 PM, Kaminski, Adi wrote: Indeed RAID10 with both mirroring and striping should satisfy the need, but per some benchmarks in the network there is still an impact on write performance on it compared to RAID0 which is considered as much better (attaching a table that summarizes

Re: SolrCloud recommended I/O RAID level

2019-07-30 Thread Kaminski, Adi
Hi Furkan, Thanks for your response ! Indeed RAID10 with both mirroring and striping should satisfy the need, but per some benchmarks in the network there is still an impact on write performance on it compared to RAID0 which is considered as much better (attaching a table that summarizes differ

Re: SolrCloud recommended I/O RAID level

2019-07-30 Thread Furkan KAMACI
Hi Adi, RAID10 is good for satisfying both indexing and query, striping across mirror sets. However, you lose half of your raw disk space, just like with RAID1. Here is a mail thread of mine which discusses RAID levels for Solr specific: https://lists.apache.org/thread.html/462d7467b2f2d064223eb4

SolrCloud recommended I/O RAID level

2019-07-29 Thread Kaminski, Adi
Hi, We are about to size large environment with 7 nodes/servers with replication factor 2 of SolrCloud cluster (using Solr 7.6). The system contains parent-child (nested documents) schema, and about to have 40M parent docs with 50-80 child docs each (in total 2-3.2B Solr docs). We have a use ca