Hi, *Problem:* write speeds are faster when no L2ARC or ZIL is configured.
*Our current setup:* We are currently running OpenIndiana b148 (upgraded from b134.) Supermicro X8DTH-i/6/iF/6F, single Xeon E5504, 24GB ram. A single, main storage pool is running pool version 28, populated with 12 WD RE 7200rpm SATA disks in a RAIDZ2. This pool has two 32GB Intel X25-E SSD's for ZIL and L2ARC connected directly to SATA ports on the motherboard. The entire system has been in operation for about one year with minimal issues. About a week ago we started seeing slow write performance so troubleshooting began. *What we have done so far / what we know:* We removed the ZIL and L2ARC SSD's from the server, zpool remove tank c6t1d0 c6t0d0 and connected them to a windows machine and ran Intel SSD Tool<http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=18455>on them. (a windows only application) Using Intel SSD Toolbox 2.0.2.000, we see the following values; 09 Power-On Hours Count: ZIL: Raw: 6783 L2ARC: Raw: 8562 E9 Media Wearout Indicator: ZIL: Raw: 0 Normalized: 99 Threshold: 0 L2ARC: Raw: 0 Normalized: 99 Threshold: 0 E1 Host Writes ZIL: Raw: 47 TB Normalized: 200 Threshold: 0 L2ARC: Raw: 67 TB Normalized: 199 Threshold: 0 Looking at this, we can only conclude that either; 1) Intel X25-E drives have no "wear" even after ~50-60 TB of writes 2) The wearout indicator is broken and unreliable. Here are some write tests we have performed using rsync to transfer a 3.5GB ISO file from my workstation over Gigabit Ethernet to a file system in this server. All tests go to the same file system unless otherwise noted, and after each test the already transfered bits were removed from the server. TRANSFER AMOUNT TIME TAKEN NOTES/CONDITIONS 1GB 6:00min tank/shares/sw which has (compression=gzip-6) X25-E ZIL and L2ARC are present 1GB 50sec rpool/home/chris (two sata disks in mirror, no compression) 1GB 3:30 tank pool without L2ARC 1GB 1:30 tank pool no L2ARC and no ZIL 500MB 4:30 tank pool, brand new ZIL, Corsair F40GB2 (40GB) 500MB 3:50 tank pool, new ZIL and new L2ARC, both Corsair F40GB2 800MB 6:00 tank pool, new L2ARC and new ZIL have now had ~3 hours to "warm up", l2arcstat went from 47MB to 22GB So our problem is that even with brand new SSD's, that have MUCH higher maximum write speeds then are "old" SSD's, transfers happen to the storage pool quicker *without* configured log and cache devices, then when they are being used. FWIW, looking at iostat -exn while running one of the rsync tests above, the time things are taking seem to match up on the kw/s column. Can anyone provide insight to this slow write speed situation when ZIL and L2ARC is present? _______________________________________________ OpenIndiana-discuss mailing list [email protected] http://openindiana.org/mailman/listinfo/openindiana-discuss
