On Fri, 2026-05-22 at 16:25 -0700, David Christensen wrote: > On 5/22/26 12:43, nwe wrote: > > On 5/22/26 1:32 PM, Van Snyder wrote: > > > > > I remarked to a local computer repair shop that I have a four TB > > > backup drive. He said "replace it. Four TB isn't ready yet." > > > > How so? I though 4TB is showing its age... > > > +1 I am also curious why 4 TB HDD's are not "ready yet".
The repair guy who made that remark to me didn't explain why he believed it. Maybe he's just not keeping up to what's happening. My WDC WD40EDAZ-11CFPB0 reports 68 power-on resets, zero read errors, and zero seek errors, but with only 2,791 power-on hours. > > I'm running 12x 4TB drives. Used SAS drives. Accumulated power on > > time > > ranges from 40,166 to 73,439 hours. > > > > Smartctl informs me device /dev/sdf is worsening with increased > > read > > errors over time. That one shows 73408 hours powered up, 72360.67 > > GB > > read, 119545.193 GB written, 195 power cycles (13 since July 13 > > 2024). > > Defect list increased from 3 to 6872. > > > > I see two other drives have defect lists of 23 and 14, > > respectively. All > > others are at 0. Considering that, I should probably prioritize > > replacing at least sdf soon to avoid losing redundancy during > > resilver, > > considering the age of the pool. > > > I am still trying to understand the smartctl(8) "SMART Attributes > Data > Structure". The RAW_VALUE seems to be a binary bit field (?) for > several attributes and is useless without manufacturer engineering > data. > The VALUE column is supposed to be a percentage that starts at 100% > and goes down to 0% as the disk wears out: > > * Raw_Read_Error_Rate, Seek_Error_Rate, and Hardware_ECC_Recovered > can > have low VALUE numbers, but the disk seems to keep working. > > * Low VALUE numbers for Reallocated_Sector_Ct, > Current_Pending_Sector, > and/or Offline_Uncorrectable seem to be reliable indicators of a > failing > disk. > > * I have not seen a VALUE number other than 100% for End-to- > End_Error. > > > > nwe@srv01:~$ zpool status -c vendor,model,size > > pool: POOL1 > > state: ONLINE > > scan: scrub repaired 0B in 04:10:05 with 0 errors on Sun May 10 > > 04:34:06 > > 2026 > > config: > > > > NAME STATE READ WRITE CKSUM vendor model size > > POOL1 ONLINE 0 0 0 > > raidz3-0 ONLINE 0 0 0 > > sdb ONLINE 0 0 0 SEAGATE ST4000NM0023 3.6T > > sdc ONLINE 0 0 0 TOSHIBA MG04SCA40EN 3.6T > > sdd ONLINE 0 0 0 TOSHIBA MG04SCA40EN 3.6T > > sde ONLINE 0 0 0 SEAGATE ST4000NM0023 3.6T > > sdf ONLINE 0 0 0 SEAGATE ST4000NM0023 3.6T > > sdh ONLINE 0 0 0 HP MB4000FCWDK 3.6T > > sdg ONLINE 0 0 0 HP MB4000FCWDK 3.6T > > sdi ONLINE 0 0 0 SEAGATE ST4000NM0023 3.6T > > sdj ONLINE 0 0 0 SEAGATE ST4000NM0023 3.6T > > sdk ONLINE 0 0 0 SEAGATE ST4000NM0023 3.6T > > sdl ONLINE 0 0 0 HP MB4000FCWDK 3.6T > > sdm ONLINE 0 0 0 HP MB4000FCWDK 3.6T > > > > errors: No known data errors > > > Twelve disks gives you many choices for how to layout the pool and > trade-off redundancy vs. capacity vs. performance. Is the data > balanced > across disks? Does the machine have enough memory? Is the ARC > working > well? > > > On two of my earlier pools, I added a 60 GB SSD as a cache vdev after > the pools had data. I did not notice any improvement. > > > On one of my earlier pools of one mirror of two 3 TB HDD's that was > nearly full, I added another mirror of two 3 TB HDD's. I did not > notice > any improvement. > > > I rebuilt the storage pool with two mirrors of two 3 TB HDD's each > and a > special vdev mirror of two 180 GB SSD's. I also set > special_small_blocks=16K. I then restored the data via replication. > The data is now balanced across disks, latency has dropped, > throughput > has increased, and overall performance is noticeably better: > > 2026-05-22 15:12:45 toor@f5 ~ > # freebsd-version > 13.5-RELEASE-p12 > > 2026-05-22 15:19:47 toor@f5 ~ > # zpool iostat -v p5 > capacity operations bandwidth > pool alloc free read write read write > ---------------- ----- ----- ----- ----- ----- ----- > p5 3.76T 1.82T 6 1 3.68M 32.2K > mirror-0 1.87T 871G 2 0 1.82M 4.48K > gpt/hdd0.eli - - 1 0 931K 2.24K > gpt/hdd1.eli - - 1 0 931K 2.24K > mirror-1 1.86T 876G 2 0 1.81M 4.35K > gpt/hdd2.eli - - 1 0 928K 2.18K > gpt/hdd3.eli - - 1 0 928K 2.18K > special - - - - - - > mirror-2 31.1G 118G 1 1 51.2K 23.3K > gpt/ssd0.eli - - 0 0 25.6K 11.7K > gpt/ssd1.eli - - 0 0 25.6K 11.7K > ---------------- ----- ----- ----- ----- ----- ----- ----- > > 2026-05-22 15:32:42 toor@f5 ~ > # top -d 1 | head -n 7 > last pid: 57622; load averages: 0.24, 0.21, 0.17 up 24+22:47:05 > 15:32:45 > 27 processes: 1 running, 26 sleeping > CPU: 0.0% user, 0.0% nice, 0.6% system, 0.0% interrupt, 99.4% > idle > Mem: 4848K Active, 330M Inact, 856K Laundry, 14G Wired, 920M Buf, > 694M Free > ARC: 12G Total, 10G MFU, 485M MRU, 3328K Anon, 200M Header, 899M > Other > 9921M Compressed, 33G Uncompressed, 3.36:1 Ratio > Swap: 764M Total, 764M Free > > 2026-05-22 15:33:12 toor@f5 ~ > # arc_summary | grep -A 5 "ARC total accesses" > ARC total accesses (hits + misses): > 512.7M > Cache hit ratio: 99.8 % > 511.8M > Cache miss ratio: 0.2 % > 886.5k > Actual hit ratio (MFU + MRU hits): 99.3 % > 509.2M > Data demand efficiency: 99.5 % > 4.8M > Data prefetch efficiency: 19.2 % > 96.9k > > > In hindsight: > > 1. I gathered file system statistics prior rebuilding the pool and > setting special_small_blocks=16K, but it now appears I could have > used a > larger value. > > 2. If I get worried about HDD's failing, I can add disks to the pool > as > spares and/or add disks to the data mirrors. The latter should > increase > read performance even more. > > 3. My ~10 year old HDD's can already saturate Gigabit with > sequential > I/O. RAID 10 with SSD acceleration is even more overkill. I want 10 > GbE. > > > David >
