On 5/22/26 17:16, nwe wrote:
On 5/22/26 6:25 PM, David Christensen wrote:
I am still trying to understand the smartctl(8) "SMART Attributes Data
Structure". The RAW_VALUE seems to be a binary bit field (?)
same here
some makes/models of hardware seem to produce a greater quantity of
comprehensible smart data
I run like
# smartctl -x /dev/sdf
returns additional data.
Twelve disks gives you many choices for how to layout the pool and
trade-off redundancy vs. capacity vs. performance. Is the data
balanced across disks? Does the machine have enough memory? Is the
ARC working well?
It has 256GB RAM.
My desktop pc currently has only 1Gb networking ever since I replaced my
fiber optic card with a gpu in the lone PCIe slot. During the time I had
10g networking direct from server to workstation, I recall easily
saturating the network. Amazing speed, but I needed the gpu more. 10g
networking is faster than the cheap SSDs in most of my PCs.
12 HDD raidz3 already exceeds 10 Gbps for sequential I/O.
256 GB of memory could be enough to fit your entire workload within the
ARC, so random I/O may have also saturated 10 Gbps. If your processor
memory bus has enough channels, your NIC has enough PCIe lanes, and your
workload does not require synchronous writes (or you tune ZFS to fake
it), a suitable workstation or backup server could saturate 25, 50, 100
Gbps single/ dual/ quad Ethernet. SFPx switches are expensive, so I
have considered dual SFPx cards in my workstation, primary server, and
backup server; connected in a ring.
If you have a USB 3.x A or C port, various manufacturers make 2.5, 5,
and 10 GbE (copper RJ-45) Ethernet adapters. If you have a Thunderbolt
3, 4, or 5 port, a few make 10 and 25 Gbps SFPx fiber single and dual
Ethernet adapters. Be sure to verify Debian and Linux driver support
with the manufacturer before purchasing:
https://www.amazon.com/s?k=thunderbolt+sfp+ethernet
https://www.sonnettech.com/product/twin25g/overview.html
I have been using Intel SSD 520 Series 2.5" SATA III drives for many
years. They are an enterprise grade product that were put in various
desktops, laptops, and netbooks ~14 years ago. Resellers harvest and
resell them on eBay. The 60 GB model works well for Linux and FreeBSD
system drives. The 180 GB model has the best performance
specifications, and works well for Windows system drives and ZFS
accelerators (at my 6 TB scale). Prices for used smaller drives can
still be reasonable in spite of the AI bubble.
All that said, when your workload goes outside the ARC or the HDD
caches, HDD latency will become the bottleneck. SATA/SAS 6 or 12 Gbps
SSD accelerators chosen to match the workload could help. NVMe PCIe
would be even better. Optane would be best:
https://goughlui.com/2024/07/28/tech-flashback-intel-optane-3d-xpoint-memory/
How do you back up a 36 TB pool?
David
