Hi Toke, Thanks for your answer.
We are using RAID 0 of 8 disk, I don't understand why it should give me the same performance as disk per drive. Below is an explanation as I see it please correct me if I'm wrong. RAID configuration each shard has data on each one of the 8 disks in the RAID, on each query to get 1K docs, each shard request to get data from the one RAID disk, so we get 8 request to get date from all of the disks and we get a queue. Shard per disk configuration each shard has data only on his own disk, each shard request to get data from his own disk and they don't block each other. If I'm wrong please correct me, I do want to get it. Regarding the response time, 2-3 seconds is good for our usage also getting better is always better, if we will get better we might run the analysis on more than 1K. Thanks for the help. NIMROD COHEN Software Engineer RTI (T) +972 (9) 775-3668 (M) +972 (0) 52-5522901 nimrod.co...@nice.com www.nice.com -----Original Message----- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: יום ג, 20 ינואר 2015 19:43 To: solr-user@lucene.apache.org Subject: RE: shards per disk Nimrod Cohen [nimrod.co...@nice.com] wrote: > We need to get 1K documents out of 100M documents each time we query > solr and send them to text Analysis. > First configuration had 8 shards on one RAD (Disk F) we got the 1K in > around 15 seconds. > Second configuration we removed the RAD and work on 8 different disk > each shard on one disk and get the 1K documents in 2-3 seconds. Which RAID level? 0, 1, maybe 5 or 6? If you did a RAID 0, it should be about the same performance as shards on individual disks, due to striping. If you did a RAID 1 with, for example, 2*4 disks, your performance would be markedly worse. If you did a RAID 1 of 8*1 disk, it would be better than individual drives as it would mitigate the "slowest drive dictates overall speed" problem. If your RAID is not really a RAID but instead JBOD or similar (http://en.wikipedia.org/wiki/Non-RAID_drive_architectures#JBOD), then the poor performance is to be expected as chances are all your data would reside on the same physical disk. Please describe your RAID setup in detail. Also, is 2-3 second response time satisfactory to you? If not, what are you aiming at? - Toke Eskildsen
