I found the write culprit, write back cache was enabled in the raid0s being presented to zfs
-----Original Message----- From: Grant Albitz Sent: Sunday, February 17, 2013 9:23 PM To: [email protected] Subject: RE: [OpenIndiana-discuss] ISCSI SSD Performance Here are some interesting results. If I put 8 ssds in a raid 10 I get about 1.2Gbps write Read is less but good at 1.1GBps, which I don't understand why its less but here is the iostat for the reads, I am listing just one drives iostat results to keep the message shorter. In all cases all of the drives are consistent with each other. r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 1120.6 0.0 142248.1 0.0 0.0 2.2 0.0 2.0 2 38 c1t0d0 If I double the ssds to 16 the writes go down drastically to 421MBps. There doesn't seem to be a specific device dragging me down. I can understand diminishing returns as far as drives go but I wouldn't expect adding drives to drastically cut the performance. r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 427.4 0.0 52047.4 0.0 9.7 0.0 22.7 1 98 c1t0d0 The reads stay the same r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 550.2 0.0 70019.3 0.0 0.0 0.9 0.0 1.6 1 25 c1t0d0 If I add all 24 drives to the raid 10 I see the following for writes: 355MB/S r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 233.2 0.0 28410.7 0.0 9.4 0.0 40.5 0 97 c1t0d0 And reads stay the same at 1.1GBps r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 369.0 0.0 47029.6 0.0 0.0 0.7 0.0 1.9 1 20 c1t0d0 Presently I have each drive in a raid 0 presented through a perc h710. I suppose it could just be tripping over itself. I originally had a h310 in the server but the drivers weren't really mature enough and I was actually having the entire system go down. Just wondering if anyone has any recommendations. At this point I could just start creating multiple pools _______________________________________________ OpenIndiana-discuss mailing list [email protected] http://openindiana.org/mailman/listinfo/openindiana-discuss
