Debugging by voodoo dolls: I made a wild guess that stumbling over cached objects was getting in the way. I dropped caches with:
echo 3 > /proc/sys/vm/drop_caches and immediately the read iops to the pool skyrocketed and the ripgrep output sure looks like it's going WAY faster: capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 75 0 403K 0 mirror 588G 340G 35 0 211K 0 sdj - - 23 0 128K 0 sdk - - 11 0 83.7K 0 mirror 588G 340G 39 0 191K 0 sdl - - 16 0 79.7K 0 sdm - - 22 0 112K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 5 0 47.8K 0 mirror 443G 2.29T 1 0 15.9K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 7.97K 0 mirror 443G 2.29T 0 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 0 0 0 mirror 443G 2.29T 2 0 23.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 1 0 15.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 3 0 16.4K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 67 0 375K 0 mirror 588G 340G 43 0 255K 0 sdj - - 25 0 147K 0 sdk - - 17 0 108K 0 mirror 588G 340G 23 0 120K 0 sdl - - 6 0 31.9K 0 sdm - - 16 0 87.7K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 3 0 15.9K 0 mirror 443G 2.29T 0 0 3.99K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 3.99K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 1 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 1 0 7.97K 0 mirror 443G 2.29T 0 0 3.99K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 0 3.99K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 0 0 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 1 0 4.98K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 54 0 279K 0 mirror 588G 340G 30 0 144K 0 sdj - - 19 0 87.7K 0 sdk - - 10 0 55.8K 0 mirror 588G 340G 23 0 136K 0 sdl - - 6 0 59.8K 0 sdm - - 16 0 75.8K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 0 12.0K 0 mirror 443G 2.29T 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 2 0 12.0K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 0 3.99K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 1 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 0 0 0 mirror 443G 2.29T 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 0 0 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 1 0 11.5K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 15.6K 0 98.7M 0 mirror 588G 340G 7.75K 0 50.0M 0 sdj - - 3.98K 0 24.8M 0 sdk - - 3.77K 0 25.1M 0 mirror 588G 340G 7.88K 0 48.8M 0 sdl - - 3.92K 0 23.8M 0 sdm - - 3.96K 0 25.0M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 418 0 3.38M 0 mirror 443G 2.29T 144 0 857K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 52 0 363K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 53 0 291K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 37 0 203K 0 mirror 443G 2.29T 100 0 1.15M 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 27 0 391K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 52 0 498K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 19 0 291K 0 mirror 443G 2.29T 173 0 1.39M 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 103 0 713K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 27 0 199K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 41 0 514K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 90 0 469K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 15.7K 0 110M 0 mirror 588G 340G 8.01K 0 57.7M 0 sdj - - 3.98K 0 29.3M 0 sdk - - 4.02K 0 28.4M 0 mirror 588G 340G 7.64K 0 52.6M 0 sdl - - 3.82K 0 26.1M 0 sdm - - 3.82K 0 26.5M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 236 0 1.12M 0 mirror 443G 2.29T 93 0 426K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 32 0 139K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 31 0 151K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 28 0 135K 0 mirror 443G 2.29T 44 0 235K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 11 0 87.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 8 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 23 0 108K 0 mirror 443G 2.29T 97 0 490K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 56 0 315K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 27 0 112K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 12 0 63.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 214 0 1.02M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 0 0 0 0 mirror 588G 340G 0 0 0 0 sdj - - 0 0 0 0 sdk - - 0 0 0 0 mirror 588G 340G 0 0 0 0 sdl - - 0 0 0 0 sdm - - 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 117 0 498K 0 mirror 443G 2.29T 50 0 215K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 15 0 67.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 14 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 19 0 83.7K 0 mirror 443G 2.29T 26 0 116K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 9 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 8 0 35.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 7 0 31.9K 0 mirror 443G 2.29T 39 0 167K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 10 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 17 0 71.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 10 0 47.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 205 0 1.04M 7.97K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 0 0 0 0 mirror 588G 340G 0 0 0 0 sdj - - 0 0 0 0 sdk - - 0 0 0 0 mirror 588G 340G 0 0 0 0 sdl - - 0 0 0 0 sdm - - 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 127 0 558K 0 mirror 443G 2.29T 51 0 231K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 16 0 71.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 9 0 51.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 24 0 108K 0 mirror 443G 2.29T 35 0 159K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 10 0 43.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 6 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 17 0 83.7K 0 mirror 443G 2.29T 39 0 167K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 18 0 79.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 11 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 8 0 39.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 202 0 989K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 12.6K 0 90.3M 0 mirror 588G 340G 6.46K 0 46.5M 0 sdj - - 3.25K 0 23.6M 0 sdk - - 3.21K 0 22.9M 0 mirror 588G 340G 6.15K 0 43.8M 0 sdl - - 3.07K 0 22.0M 0 sdm - - 3.08K 0 21.9M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 129 0 550K 0 mirror 443G 2.29T 47 0 199K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 18 0 83.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 15 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 12 0 51.8K 0 mirror 443G 2.29T 28 0 131K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 3 0 15.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 11 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 12 0 67.7K 0 mirror 443G 2.29T 52 0 219K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 25 0 112K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 15 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 10 0 43.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 146 0 715K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 7.97K 0 54.7M 0 mirror 588G 340G 3.77K 0 26.1M 0 sdj - - 1.92K 0 13.1M 0 sdk - - 1.85K 0 13.0M 0 mirror 588G 340G 4.20K 0 28.6M 0 sdl - - 2.14K 0 13.7M 0 sdm - - 2.06K 0 14.9M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 112 0 482K 0 mirror 443G 2.29T 36 0 155K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 11 0 51.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 10 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 13 0 59.7K 0 mirror 443G 2.29T 25 0 115K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 9 0 39.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 8 0 39.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 6 0 35.8K 0 mirror 443G 2.29T 49 0 211K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 26 0 111K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 14 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 7 0 35.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 170 1 880K 15.9K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 8.45K 0 59.9M 0 mirror 588G 340G 4.26K 0 30.4M 0 sdj - - 2.13K 0 15.5M 0 sdk - - 2.13K 0 15.0M 0 mirror 588G 340G 4.19K 0 29.4M 0 sdl - - 2.16K 0 15.0M 0 sdm - - 2.03K 0 14.4M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 87 0 394K 0 mirror 443G 2.29T 27 0 128K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 10 0 55.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 7 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 8 0 39.8K 0 mirror 443G 2.29T 30 0 139K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 4 0 23.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 16 0 67.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 8 0 47.8K 0 mirror 443G 2.29T 28 0 128K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 6 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 11 0 55.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 9 0 43.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 174 0 839K 7.97K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 11.3K 0 82.7M 0 mirror 588G 340G 5.56K 0 39.6M 0 sdj - - 2.79K 0 19.7M 0 sdk - - 2.77K 0 19.9M 0 mirror 588G 340G 5.71K 0 43.1M 0 sdl - - 2.92K 0 22.1M 0 sdm - - 2.79K 0 21.1M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 584 495 2.83M 2.75M mirror 443G 2.29T 178 172 893K 981K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 59 60 347K 327K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 55 61 243K 327K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 62 49 303K 327K mirror 443G 2.29T 183 172 849K 981K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 69 58 299K 327K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 83 59 411K 327K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 29 53 140K 327K mirror 443G 2.29T 222 150 1.13M 849K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 60 50 331K 283K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 85 51 387K 283K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 75 47 435K 283K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 112 0 598K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 3.50K 0 24.4M 0 mirror 588G 340G 1.74K 0 12.1M 0 sdj - - 898 0 6.03M 0 sdk - - 880 0 6.08M 0 mirror 588G 340G 1.76K 0 12.3M 0 sdl - - 895 0 6.30M 0 sdm - - 906 0 5.98M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 74 0 323K 0 mirror 443G 2.29T 24 0 108K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 4 0 19.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 8 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 10 0 47.8K 0 mirror 443G 2.29T 19 0 87.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 5 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 6 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 6 0 31.9K 0 mirror 443G 2.29T 29 0 128K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 16 0 71.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 7 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 4 0 23.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 143 1 815K 15.9K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 2.87K 0 23.3M 0 mirror 588G 340G 1.38K 0 12.3M 0 sdj - - 673 0 5.89M 0 sdk - - 735 0 6.40M 0 mirror 588G 340G 1.49K 0 11.0M 0 sdl - - 758 0 5.96M 0 sdm - - 767 0 5.09M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 534 0 2.35M 0 mirror 443G 2.29T 141 0 733K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 57 0 263K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 45 0 259K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 37 0 211K 0 mirror 443G 2.29T 156 0 653K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 48 0 207K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 78 0 323K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 28 0 123K 0 mirror 443G 2.29T 236 0 1023K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 93 0 410K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 130 0 558K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 12 0 55.7K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 163 0 761K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 4 0 19.9K 0 mirror 588G 340G 1 0 7.96K 0 sdj - - 1 0 7.96K 0 sdk - - 0 0 0 0 mirror 588G 340G 2 0 11.9K 0 sdl - - 0 0 3.98K 0 sdm - - 1 0 7.96K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 50 0 362K 0 mirror 443G 2.29T 26 0 163K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 4 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 11 0 83.6K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 9 0 51.8K 0 mirror 443G 2.29T 8 0 51.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 3 0 23.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 3 0 19.9K 0 mirror 443G 2.29T 14 0 147K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 5 0 83.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 3 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 4 0 35.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 14 0 72.7K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 8.64K 0 62.3M 0 mirror 588G 340G 4.39K 0 31.7M 0 sdj - - 2.22K 0 16.2M 0 sdk - - 2.18K 0 15.6M 0 mirror 588G 340G 4.25K 0 30.6M 0 sdl - - 2.15K 0 15.4M 0 sdm - - 2.10K 0 15.2M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 124 0 805K 0 mirror 443G 2.29T 53 0 271K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 18 0 87.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 13 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 20 0 120K 0 mirror 443G 2.29T 29 0 147K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 11 0 55.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 8 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 8 0 43.8K 0 mirror 443G 2.29T 40 0 387K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 22 0 307K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 8 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 8 0 39.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 118 0 649K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 6.73K 0 44.6M 0 mirror 588G 340G 3.20K 0 21.4M 0 sdj - - 1.61K 0 10.4M 0 sdk - - 1.59K 0 11.0M 0 mirror 588G 340G 3.53K 0 23.2M 0 sdl - - 1.82K 0 11.7M 0 sdm - - 1.72K 0 11.5M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 199 0 1.23M 0 mirror 443G 2.29T 55 0 335K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 17 0 144K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 16 0 83.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 20 0 108K 0 mirror 443G 2.29T 62 0 403K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 30 0 140K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 11 0 175K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 19 0 87.7K 0 mirror 443G 2.29T 80 0 526K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 50 0 243K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 17 0 175K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 11 0 108K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 202 0 999K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 10.8K 0 65.4M 0 mirror 588G 340G 5.45K 0 32.7M 0 sdj - - 2.82K 0 16.7M 0 sdk - - 2.63K 0 15.9M 0 mirror 588G 340G 5.38K 0 32.7M 0 sdl - - 2.74K 0 16.5M 0 sdm - - 2.64K 0 16.2M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 236 0 2.09M 0 mirror 443G 2.29T 85 0 470K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 36 0 195K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 24 0 124K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 23 0 151K 0 mirror 443G 2.29T 93 0 773K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 42 0 243K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 10 0 104K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 39 0 426K 0 mirror 443G 2.29T 56 0 901K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 27 0 482K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 9 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 18 0 375K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 138 0 607K 7.97K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 13.4K 0 167M 0 mirror 588G 340G 6.94K 0 85.5M 0 sdj - - 3.48K 0 43.6M 0 sdk - - 3.46K 0 41.8M 0 mirror 588G 340G 6.50K 0 81.4M 0 sdl - - 3.26K 0 41.5M 0 sdm - - 3.24K 0 39.9M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 112 0 502K 0 mirror 443G 2.29T 35 0 151K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 13 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 6 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 14 0 63.7K 0 mirror 443G 2.29T 33 0 167K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 9 0 55.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 11 0 51.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 11 0 59.8K 0 mirror 443G 2.29T 42 0 183K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 21 0 95.6K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 6 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 13 0 59.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 246 0 1.11M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 10.5K 0 365M 0 mirror 588G 340G 5.24K 0 184M 0 sdj - - 2.56K 0 89.6M 0 sdk - - 2.68K 0 94.7M 0 mirror 588G 340G 5.29K 0 181M 0 sdl - - 2.76K 0 94.5M 0 sdm - - 2.52K 0 86.4M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 109 0 494K 0 mirror 443G 2.29T 27 0 124K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 5 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 13 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 7 0 31.9K 0 mirror 443G 2.29T 42 0 203K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 12 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 14 0 71.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 14 0 71.8K 0 mirror 443G 2.29T 38 0 167K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 24 0 104K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 6 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 6 0 31.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 373 0 1.80M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 19.2K 0 94.0M 0 mirror 588G 340G 9.77K 0 47.2M 0 sdj - - 4.93K 0 24.0M 0 sdk - - 4.84K 0 23.2M 0 mirror 588G 340G 9.45K 0 46.8M 0 sdl - - 4.75K 0 23.5M 0 sdm - - 4.69K 0 23.3M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 65 0 295K 0 mirror 443G 2.29T 7 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 2 0 15.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 1 0 7.97K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 2 0 15.9K 0 mirror 443G 2.29T 29 0 136K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 8 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 12 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 7 0 35.9K 0 mirror 443G 2.29T 27 0 120K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 19 0 83.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 2 0 12.0K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 4 0 23.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 162 0 711K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 12.6K 0 94.8M 0 mirror 588G 340G 7.28K 0 54.5M 0 sdj - - 3.68K 0 26.2M 0 sdk - - 3.60K 0 28.2M 0 mirror 588G 340G 5.33K 0 40.3M 0 sdl - - 2.61K 0 19.8M 0 sdm - - 2.72K 0 20.5M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 129 548 554K 5.98M mirror 443G 2.29T 28 214 120K 4.16M ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 10 66 43.9K 1.39M ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 9 74 39.9K 1.39M ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 7 72 35.9K 1.39M mirror 443G 2.29T 53 181 235K 1.05M ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 26 57 116K 359K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 17 62 75.8K 359K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 8 60 43.9K 359K mirror 443G 2.29T 46 152 199K 790K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 25 48 108K 263K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 11 48 51.8K 263K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 8 54 39.9K 263K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 272 0 1.10M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 14.3K 0 104M 0 mirror 588G 340G 7.26K 0 54.0M 0 sdj - - 3.66K 0 26.9M 0 sdk - - 3.60K 0 27.2M 0 mirror 588G 340G 7.05K 0 49.6M 0 sdl - - 3.49K 0 23.8M 0 sdm - - 3.56K 0 25.8M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 111 0 482K 0 mirror 443G 2.29T 25 0 116K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 9 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 7 0 35.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 7 0 35.9K 0 mirror 443G 2.29T 49 0 219K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 15 0 71.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 18 0 75.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 14 0 71.7K 0 mirror 443G 2.29T 35 0 147K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 20 0 83.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 4 0 23.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 9 0 39.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 216 0 1.04M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 14.7K 0 105M 0 mirror 588G 340G 9.21K 0 66.1M 0 sdj - - 4.61K 0 33.2M 0 sdk - - 4.60K 0 32.9M 0 mirror 588G 340G 5.49K 0 39.3M 0 sdl - - 2.73K 0 19.3M 0 sdm - - 2.76K 0 20.1M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 98 0 438K 0 mirror 443G 2.29T 25 0 120K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 13 0 71.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 4 0 19.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 6 0 27.9K 0 mirror 443G 2.29T 43 0 199K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 20 0 91.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 7 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 14 0 75.7K 0 mirror 443G 2.29T 28 0 120K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 12 0 55.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 7 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 7 0 31.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 206 0 990K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 13.8K 0 92.5M 0 mirror 588G 340G 6.88K 0 46.9M 0 sdj - - 3.57K 0 23.6M 0 sdk - - 3.31K 0 23.4M 0 mirror 588G 340G 6.91K 0 45.6M 0 sdl - - 3.50K 0 23.2M 0 sdm - - 3.41K 0 22.4M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 104 0 467K 0 mirror 443G 2.29T 34 0 155K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 9 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 11 0 51.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 12 0 55.8K 0 mirror 443G 2.29T 30 0 136K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 14 0 67.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 7 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 7 0 35.9K 0 mirror 443G 2.29T 38 0 175K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 18 0 79.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 7 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 11 0 55.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 245 0 1.03M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 13.5K 0 127M 0 mirror 588G 340G 6.54K 0 60.4M 0 sdj - - 3.29K 0 30.6M 0 sdk - - 3.25K 0 29.8M 0 mirror 588G 340G 6.95K 0 67.1M 0 sdl - - 3.50K 0 33.8M 0 sdm - - 3.46K 0 33.4M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 101 0 434K 0 mirror 443G 2.29T 41 0 171K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 17 0 75.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 9 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 13 0 55.8K 0 mirror 443G 2.29T 23 0 108K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 9 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 6 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 6 0 31.9K 0 mirror 443G 2.29T 35 0 155K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 13 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 7 0 31.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 13 0 63.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 265 0 1.16M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 14.7K 0 94.1M 0 mirror 588G 340G 7.37K 0 46.2M 0 sdj - - 3.73K 0 22.9M 0 sdk - - 3.64K 0 23.2M 0 mirror 588G 340G 7.27K 0 47.9M 0 sdl - - 3.75K 0 24.8M 0 sdm - - 3.52K 0 23.1M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 107 0 466K 0 mirror 443G 2.29T 48 0 211K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 9 0 39.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 17 0 79.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 20 0 91.7K 0 mirror 443G 2.29T 29 0 124K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 10 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 11 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 6 0 27.9K 0 mirror 443G 2.29T 28 0 132K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 9 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 12 0 55.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 5 0 31.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 137 0 644K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 21.1K 0 129M 0 mirror 588G 340G 10.7K 0 64.1M 0 sdj - - 5.41K 0 32.1M 0 sdk - - 5.25K 0 32.0M 0 mirror 588G 340G 10.4K 0 64.6M 0 sdl - - 5.32K 0 33.4M 0 sdm - - 5.09K 0 31.2M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 372 0 4.05M 0 mirror 443G 2.29T 146 0 1019K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 47 0 263K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 51 0 291K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 46 0 466K 0 mirror 443G 2.29T 79 0 1.06M 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 25 0 263K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 18 0 573K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 34 0 251K 0 mirror 443G 2.29T 146 0 2.00M 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 50 0 1.38M 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 64 0 279K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 30 0 350K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 164 0 737K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 20.2K 0 251M 0 mirror 588G 340G 10.6K 0 131M 0 sdj - - 5.19K 0 63.8M 0 sdk - - 5.39K 0 66.8M 0 mirror 588G 340G 9.63K 0 121M 0 sdl - - 4.79K 0 59.4M 0 sdm - - 4.84K 0 61.5M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 102 0 450K 0 mirror 443G 2.29T 53 0 247K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 12 0 51.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 13 0 59.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 26 0 135K 0 mirror 443G 2.29T 31 0 127K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 14 0 59.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 8 0 35.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 7 0 31.8K 0 mirror 443G 2.29T 16 0 75.6K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 2 0 11.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 5 0 23.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 7 0 39.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 143 0 527K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 22.3K 0 275M 0 mirror 588G 340G 11.2K 0 133M 0 sdj - - 5.64K 0 66.6M 0 sdk - - 5.56K 0 66.6M 0 mirror 588G 340G 11.1K 0 142M 0 sdl - - 5.62K 0 70.2M 0 sdm - - 5.49K 0 71.5M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 82 0 382K 0 mirror 443G 2.29T 25 0 123K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 9 0 47.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 6 0 35.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 8 0 39.8K 0 mirror 443G 2.29T 25 0 131K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 11 0 55.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 6 0 35.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 6 0 39.8K 0 mirror 443G 2.29T 30 0 127K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 7 0 35.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 11 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 10 0 43.8K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 172 2 731K 19.9K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 27.2K 0 178M 0 mirror 588G 340G 13.8K 0 90.7M 0 sdj - - 6.91K 0 45.4M 0 sdk - - 6.87K 0 45.3M 0 mirror 588G 340G 13.4K 0 87.4M 0 sdl - - 6.74K 0 43.9M 0 sdm - - 6.68K 0 43.6M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 111 0 669K 0 mirror 443G 2.29T 31 0 171K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 11 0 79.6K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 9 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 9 0 43.8K 0 mirror 443G 2.29T 54 0 338K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 14 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 7 0 31.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 31 0 243K 0 mirror 443G 2.29T 24 0 159K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 4 0 23.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 10 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 8 0 91.5K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 167 2 612K 19.9K ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 30.7K 0 220M 0 mirror 588G 340G 14.7K 0 105M 0 sdj - - 7.41K 0 53.7M 0 sdk - - 7.29K 0 51.7M 0 mirror 588G 340G 15.9K 0 114M 0 sdl - - 8.04K 0 57.4M 0 sdm - - 7.91K 0 57.0M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 91 387 386K 6.15M mirror 443G 2.29T 30 139 131K 1.03M ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 11 46 51.8K 350K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 9 47 43.8K 350K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 8 44 35.8K 350K mirror 443G 2.29T 28 109 127K 860K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 6 35 27.9K 287K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 6 36 31.9K 287K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 14 36 67.7K 287K mirror 443G 2.29T 31 138 127K 4.28M ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 15 47 63.7K 1.43M ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 4 43 19.9K 1.43M ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 10 46 43.8K 1.43M logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 126 0 560K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 25.0K 0 206M 0 mirror 588G 340G 12.3K 0 103M 0 sdj - - 6.21K 0 51.9M 0 sdk - - 6.13K 0 50.7M 0 mirror 588G 340G 12.6K 0 103M 0 sdl - - 6.31K 0 51.6M 0 sdm - - 6.30K 0 51.5M 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 83 0 366K 0 mirror 443G 2.29T 26 0 123K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 9 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 5 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 10 0 51.8K 0 mirror 443G 2.29T 30 0 135K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 10 0 47.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 9 0 43.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 9 0 43.8K 0 mirror 443G 2.29T 25 0 108K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 5 0 27.9K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 12 0 51.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 6 0 27.9K 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 198 2 770K 15.9K ------------------------------------------- ----- ----- ----- ----- ----- ----- -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1814983 Title: zfs poor sustained read performance from ssd pool Status in zfs-linux package in Ubuntu: New Bug description: Hello, I'm seeing substantially slower read performance from an ssd pool than I expected. I have two pools on this computer; one ('fst') is four sata ssds, the other ('srv') is nine spinning metal drives. With a long-running ripgrep process on the fst pool, performance started out really good and grew to astonishingly good (iirc ~30kiops, as measured by zpool iostat -v 1). However after a few hours the performance has dropped to 30-40 iops. top reports an arc_reclaim and many arc_prune processes to be consuming most of the CPU time. I've included a screenshot of top, some output from zpool iostat -v 1, and arc_summary, with "===" to indicate the start of the next command's output: === top (memory in gigabytes): top - 16:27:53 up 70 days, 16:03, 3 users, load average: 35.67, 35.81, 35.58 Tasks: 809 total, 19 running, 612 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 58.1 sy, 0.0 ni, 39.2 id, 2.6 wa, 0.0 hi, 0.0 si, 0.0 st GiB Mem : 125.805 total, 0.620 free, 96.942 used, 28.243 buff/cache GiB Swap: 5.694 total, 5.688 free, 0.006 used. 27.840 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1523 root 20 0 0.0m 0.0m 0.0m R 100.0 0.0 290:52.26 arc_reclaim 4484 root 20 0 0.0m 0.0m 0.0m R 56.2 0.0 1:18.79 arc_prune 6225 root 20 0 0.0m 0.0m 0.0m R 56.2 0.0 1:11.92 arc_prune 7601 root 20 0 0.0m 0.0m 0.0m S 56.2 0.0 2:50.25 arc_prune 30891 root 20 0 0.0m 0.0m 0.0m S 56.2 0.0 1:33.08 arc_prune 3057 root 20 0 0.0m 0.0m 0.0m S 55.9 0.0 9:00.95 arc_prune 3259 root 20 0 0.0m 0.0m 0.0m R 55.9 0.0 3:16.84 arc_prune 24008 root 20 0 0.0m 0.0m 0.0m S 55.9 0.0 1:55.71 arc_prune 1285 root 20 0 0.0m 0.0m 0.0m R 55.6 0.0 3:20.52 arc_prune 5345 root 20 0 0.0m 0.0m 0.0m R 55.6 0.0 1:15.99 arc_prune 30121 root 20 0 0.0m 0.0m 0.0m S 55.6 0.0 1:35.50 arc_prune 31192 root 20 0 0.0m 0.0m 0.0m S 55.6 0.0 6:17.16 arc_prune 32287 root 20 0 0.0m 0.0m 0.0m S 55.6 0.0 1:28.02 arc_prune 32625 root 20 0 0.0m 0.0m 0.0m R 55.6 0.0 1:27.34 arc_prune 22572 root 20 0 0.0m 0.0m 0.0m S 55.3 0.0 10:02.92 arc_prune 31989 root 20 0 0.0m 0.0m 0.0m R 55.3 0.0 1:28.03 arc_prune 3353 root 20 0 0.0m 0.0m 0.0m R 54.9 0.0 8:58.81 arc_prune 10252 root 20 0 0.0m 0.0m 0.0m R 54.9 0.0 2:36.37 arc_prune 1522 root 20 0 0.0m 0.0m 0.0m S 53.9 0.0 158:42.45 arc_prune 3694 root 20 0 0.0m 0.0m 0.0m R 53.9 0.0 1:20.79 arc_prune 13394 root 20 0 0.0m 0.0m 0.0m R 53.9 0.0 10:35.78 arc_prune 24592 root 20 0 0.0m 0.0m 0.0m R 53.9 0.0 1:54.19 arc_prune 25859 root 20 0 0.0m 0.0m 0.0m S 53.9 0.0 1:51.71 arc_prune 8194 root 20 0 0.0m 0.0m 0.0m S 53.6 0.0 0:54.51 arc_prune 18472 root 20 0 0.0m 0.0m 0.0m R 53.6 0.0 2:08.73 arc_prune 29525 root 20 0 0.0m 0.0m 0.0m R 53.6 0.0 1:35.81 arc_prune 32291 root 20 0 0.0m 0.0m 0.0m S 53.6 0.0 1:28.00 arc_prune 3156 root 20 0 0.0m 0.0m 0.0m R 53.3 0.0 3:17.68 arc_prune 6224 root 20 0 0.0m 0.0m 0.0m S 53.3 0.0 1:11.80 arc_prune 9788 root 20 0 0.0m 0.0m 0.0m S 53.3 0.0 0:46.00 arc_prune 10341 root 20 0 0.0m 0.0m 0.0m R 53.3 0.0 2:36.23 arc_prune 11881 root 20 0 0.0m 0.0m 0.0m S 53.0 0.0 2:31.57 arc_prune 24030 root 20 0 0.0m 0.0m 0.0m R 52.6 0.0 1:55.44 arc_prune === zpool iostat -v 1 output (for a while): capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 68 0 650K 0 mirror 588G 340G 31 0 331K 0 sdj - - 20 0 179K 0 sdk - - 10 0 152K 0 mirror 588G 340G 36 0 319K 0 sdl - - 17 0 132K 0 sdm - - 18 0 187K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 35 187K 144K mirror 443G 2.29T 0 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 0 17 0 71.8K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 5 0 23.9K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 5 0 23.9K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 5 0 23.9K mirror 443G 2.29T 1 17 124K 71.8K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 5 124K 23.9K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 5 0 23.9K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 5 0 23.9K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 110 0 1.07M 0 mirror 588G 340G 59 0 634K 0 sdj - - 28 0 303K 0 sdk - - 30 0 331K 0 mirror 588G 340G 50 0 459K 0 sdl - - 28 0 303K 0 sdm - - 21 0 155K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 229 183K 1.00M mirror 443G 2.29T 2 73 183K 335K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 2 24 183K 112K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 24 0 112K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 23 0 112K mirror 443G 2.29T 0 77 0 347K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 25 0 116K mirror 443G 2.29T 0 77 0 347K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 25 0 116K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 29 0 403K 0 mirror 588G 340G 12 0 171K 0 sdj - - 7 0 79.7K 0 sdk - - 4 0 91.7K 0 mirror 588G 340G 16 0 231K 0 sdl - - 6 0 128K 0 sdm - - 9 0 104K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 0 66 63.8K 359K mirror 443G 2.29T 0 21 0 120K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 6 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 7 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 6 0 39.9K mirror 443G 2.29T 0 21 0 120K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 7 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 6 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 6 0 39.9K mirror 443G 2.29T 0 22 63.8K 120K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 7 63.8K 39.9K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 6 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 7 0 39.9K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 97 0 797K 0 mirror 588G 340G 58 0 474K 0 sdj - - 27 0 263K 0 sdk - - 30 0 211K 0 mirror 588G 340G 38 0 323K 0 sdl - - 23 0 203K 0 sdm - - 14 0 120K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 176 187K 789K mirror 443G 2.29T 0 58 59.8K 263K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 19 59.8K 87.7K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 18 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 19 0 87.7K mirror 443G 2.29T 0 59 0 263K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 19 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 19 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 19 0 87.7K mirror 443G 2.29T 1 57 128K 263K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 18 128K 87.7K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 19 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 18 0 87.7K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 70 0 426K 0 mirror 588G 340G 38 0 263K 0 sdj - - 21 0 128K 0 sdk - - 16 0 135K 0 mirror 588G 340G 31 0 163K 0 sdl - - 10 0 67.7K 0 sdm - - 20 0 95.6K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 1 46 116K 2.36M mirror 443G 2.29T 0 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 0 37 0 2.31M ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 12 0 789K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 11 0 789K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 12 0 789K mirror 443G 2.29T 0 8 55.8K 47.8K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 2 55.8K 15.9K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 2 0 15.9K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 2 0 15.9K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 108 0 614K 0 mirror 588G 340G 50 0 299K 0 sdj - - 32 0 203K 0 sdk - - 17 0 95.6K 0 mirror 588G 340G 57 0 315K 0 sdl - - 30 0 155K 0 sdm - - 26 0 159K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 68 191K 311K mirror 443G 2.29T 0 8 0 47.8K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 2 0 15.9K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 2 0 15.9K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 2 0 15.9K mirror 443G 2.29T 0 29 0 132K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 9 0 43.8K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 9 0 43.8K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 9 0 43.8K mirror 443G 2.29T 2 29 191K 132K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 2 9 191K 43.8K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 9 0 43.8K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 9 0 43.8K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 66 0 379K 0 mirror 588G 340G 26 0 144K 0 sdj - - 12 0 63.8K 0 sdk - - 13 0 79.7K 0 mirror 588G 340G 39 0 235K 0 sdl - - 19 0 120K 0 sdm - - 19 0 116K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 166 183K 754K mirror 443G 2.29T 0 55 0 251K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 18 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 17 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 18 0 83.7K mirror 443G 2.29T 0 54 0 251K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 17 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 18 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 17 0 83.7K mirror 443G 2.29T 2 55 183K 251K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 2 18 183K 83.7K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 17 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 18 0 83.7K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 126 0 698K 0 mirror 588G 340G 64 0 335K 0 sdj - - 37 0 195K 0 sdk - - 26 0 140K 0 mirror 588G 340G 61 0 363K 0 sdl - - 34 0 207K 0 sdm - - 26 0 155K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 3 274 239K 1.23M mirror 443G 2.29T 1 91 120K 418K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 1 30 120K 139K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 29 0 139K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 30 0 139K mirror 443G 2.29T 0 91 0 418K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 30 0 139K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 29 0 139K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 30 0 139K mirror 443G 2.29T 1 91 120K 418K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 30 120K 139K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 30 0 139K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 29 0 139K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 70 0 442K 0 mirror 588G 340G 36 0 215K 0 sdj - - 18 0 95.6K 0 sdk - - 17 0 119K 0 mirror 588G 340G 33 0 227K 0 sdl - - 15 0 123K 0 sdm - - 17 0 104K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 0 187K 0 mirror 443G 2.29T 0 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 0 0 0 mirror 443G 2.29T 1 0 123K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 0 123K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 0 0 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- === arc_summary ------------------------------------------------------------------------ ZFS Subsystem Report Wed Feb 06 16:34:09 2019 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 142.80m Mutex Misses: 14.89m Evict Skips: 21.87b ARC Size: 86.88% 54.65 GiB Target Size: (Adaptive) 78.61% 49.45 GiB Min Size (Hard Limit): 6.25% 3.93 GiB Max Size (High Water): 16:1 62.90 GiB ARC Size Breakdown: Recently Used Cache Size: 25.05% 5.55 GiB Frequently Used Cache Size: 74.95% 16.62 GiB ARC Hash Breakdown: Elements Max: 7.46m Elements Current: 60.33% 4.50m Collisions: 43.79m Chain Max: 8 Chains: 504.81k ARC Total accesses: 2.51b Cache Hit Ratio: 95.24% 2.39b Cache Miss Ratio: 4.76% 119.32m Actual Hit Ratio: 92.99% 2.33b Data Demand Efficiency: 94.16% 486.53m Data Prefetch Efficiency: 86.47% 29.68m CACHE HITS BY CACHE LIST: Anonymously Used: 2.20% 52.51m Most Recently Used: 37.35% 891.38m Most Frequently Used: 60.29% 1.44b Most Recently Used Ghost: 0.10% 2.32m Most Frequently Used Ghost: 0.06% 1.45m CACHE HITS BY DATA TYPE: Demand Data: 19.20% 458.13m Prefetch Data: 1.08% 25.66m Demand Metadata: 78.22% 1.87b Prefetch Metadata: 1.51% 36.00m CACHE MISSES BY DATA TYPE: Demand Data: 23.80% 28.40m Prefetch Data: 3.37% 4.02m Demand Metadata: 66.03% 78.79m Prefetch Metadata: 6.80% 8.12m L2 ARC Summary: (HEALTHY) Low Memory Aborts: 233 Free on Write: 27.52k R/W Clashes: 0 Bad Checksums: 0 IO Errors: 0 L2 ARC Size: (Adaptive) 364.94 GiB Compressed: 91.59% 334.23 GiB Header Size: 0.08% 307.98 MiB L2 ARC Breakdown: 119.32m Hit Ratio: 1.42% 1.69m Miss Ratio: 98.58% 117.63m Feeds: 6.01m L2 ARC Writes: Writes Sent: 100.00% 279.55k DMU Prefetch Efficiency: 1.89b Hit Ratio: 2.24% 42.49m Miss Ratio: 97.76% 1.85b ZFS Tunable: dbuf_cache_hiwater_pct 10 dbuf_cache_lowater_pct 10 dbuf_cache_max_bytes 104857600 dbuf_cache_max_shift 5 dmu_object_alloc_chunk_shift 7 ignore_hole_birth 1 l2arc_feed_again 1 l2arc_feed_min_ms 200 l2arc_feed_secs 1 l2arc_headroom 2 l2arc_headroom_boost 200 l2arc_noprefetch 1 l2arc_norw 0 l2arc_write_boost 8388608 l2arc_write_max 8388608 metaslab_aliquot 524288 metaslab_bias_enabled 1 metaslab_debug_load 0 metaslab_debug_unload 0 metaslab_fragmentation_factor_enabled 1 metaslab_lba_weighting_enabled 1 metaslab_preload_enabled 1 metaslabs_per_vdev 200 send_holes_without_birth_time 1 spa_asize_inflation 24 spa_config_path /etc/zfs/zpool.cache spa_load_verify_data 1 spa_load_verify_maxinflight 10000 spa_load_verify_metadata 1 spa_slop_shift 5 zfetch_array_rd_sz 1048576 zfetch_max_distance 8388608 zfetch_max_streams 8 zfetch_min_sec_reap 2 zfs_abd_scatter_enabled 1 zfs_abd_scatter_max_order 10 zfs_admin_snapshot 1 zfs_arc_average_blocksize 8192 zfs_arc_dnode_limit 0 zfs_arc_dnode_limit_percent 10 zfs_arc_dnode_reduce_percent 10 zfs_arc_grow_retry 0 zfs_arc_lotsfree_percent 10 zfs_arc_max 0 zfs_arc_meta_adjust_restarts 4096 zfs_arc_meta_limit 0 zfs_arc_meta_limit_percent 75 zfs_arc_meta_min 0 zfs_arc_meta_prune 10000 zfs_arc_meta_strategy 1 zfs_arc_min 0 zfs_arc_min_prefetch_lifespan 0 zfs_arc_p_aggressive_disable 1 zfs_arc_p_dampener_disable 1 zfs_arc_p_min_shift 0 zfs_arc_pc_percent 0 zfs_arc_shrink_shift 0 zfs_arc_sys_free 0 zfs_autoimport_disable 1 zfs_compressed_arc_enabled 1 zfs_dbgmsg_enable 0 zfs_dbgmsg_maxsize 4194304 zfs_dbuf_state_index 0 zfs_deadman_checktime_ms 5000 zfs_deadman_enabled 1 zfs_deadman_synctime_ms 1000000 zfs_dedup_prefetch 0 zfs_delay_min_dirty_percent 60 zfs_delay_scale 500000 zfs_delete_blocks 20480 zfs_dirty_data_max 4294967296 zfs_dirty_data_max_max 4294967296 zfs_dirty_data_max_max_percent 25 zfs_dirty_data_max_percent 10 zfs_dirty_data_sync 67108864 zfs_dmu_offset_next_sync 0 zfs_expire_snapshot 300 zfs_flags 0 zfs_free_bpobj_enabled 1 zfs_free_leak_on_eio 0 zfs_free_max_blocks 100000 zfs_free_min_time_ms 1000 zfs_immediate_write_sz 32768 zfs_max_recordsize 1048576 zfs_mdcomp_disable 0 zfs_metaslab_fragmentation_threshold 70 zfs_metaslab_segment_weight_enabled 1 zfs_metaslab_switch_threshold 2 zfs_mg_fragmentation_threshold 85 zfs_mg_noalloc_threshold 0 zfs_multihost_fail_intervals 5 zfs_multihost_history 0 zfs_multihost_import_intervals 10 zfs_multihost_interval 1000 zfs_multilist_num_sublists 0 zfs_no_scrub_io 0 zfs_no_scrub_prefetch 0 zfs_nocacheflush 0 zfs_nopwrite_enabled 1 zfs_object_mutex_size 64 zfs_pd_bytes_max 52428800 zfs_per_txg_dirty_frees_percent 30 zfs_prefetch_disable 0 zfs_read_chunk_size 1048576 zfs_read_history 0 zfs_read_history_hits 0 zfs_recover 0 zfs_resilver_delay 2 zfs_resilver_min_time_ms 3000 zfs_scan_idle 50 zfs_scan_min_time_ms 1000 zfs_scrub_delay 4 zfs_send_corrupt_data 0 zfs_sync_pass_deferred_free 2 zfs_sync_pass_dont_compress 5 zfs_sync_pass_rewrite 2 zfs_sync_taskq_batch_pct 75 zfs_top_maxinflight 32 zfs_txg_history 0 zfs_txg_timeout 5 zfs_vdev_aggregation_limit 131072 zfs_vdev_async_read_max_active 3 zfs_vdev_async_read_min_active 1 zfs_vdev_async_write_active_max_dirty_percent 60 zfs_vdev_async_write_active_min_dirty_percent 30 zfs_vdev_async_write_max_active 10 zfs_vdev_async_write_min_active 2 zfs_vdev_cache_bshift 16 zfs_vdev_cache_max 16384 zfs_vdev_cache_size 0 zfs_vdev_max_active 1000 zfs_vdev_mirror_non_rotating_inc 0 zfs_vdev_mirror_non_rotating_seek_inc 1 zfs_vdev_mirror_rotating_inc 0 zfs_vdev_mirror_rotating_seek_inc 5 zfs_vdev_mirror_rotating_seek_offset 1048576 zfs_vdev_queue_depth_pct 1000 zfs_vdev_raidz_impl [fastest] original scalar sse2 ssse3 avx2 zfs_vdev_read_gap_limit 32768 zfs_vdev_scheduler noop zfs_vdev_scrub_max_active 2 zfs_vdev_scrub_min_active 1 zfs_vdev_sync_read_max_active 10 zfs_vdev_sync_read_min_active 10 zfs_vdev_sync_write_max_active 10 zfs_vdev_sync_write_min_active 10 zfs_vdev_write_gap_limit 4096 zfs_zevent_cols 80 zfs_zevent_console 0 zfs_zevent_len_max 512 zfs_zil_clean_taskq_maxalloc 1048576 zfs_zil_clean_taskq_minalloc 1024 zfs_zil_clean_taskq_nthr_pct 100 zil_replay_disable 0 zil_slog_bulk 786432 zio_delay_max 30000 zio_dva_throttle_enabled 1 zio_requeue_io_start_cut_in_line 1 zio_taskq_batch_pct 75 zvol_inhibit_dev 0 zvol_major 230 zvol_max_discard_blocks 16384 zvol_prefetch_bytes 131072 zvol_request_sync 0 zvol_threads 32 zvol_volmode 1 Thanks ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: zfsutils-linux 0.7.5-1ubuntu16.4 ProcVersionSignature: Ubuntu 4.15.0-39.42-generic 4.15.18 Uname: Linux 4.15.0-39-generic x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair ApportVersion: 2.20.9-0ubuntu7.5 Architecture: amd64 Date: Wed Feb 6 16:26:17 2019 InstallationDate: Installed on 2016-04-04 (1038 days ago) InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Beta amd64 (20160325) ProcEnviron: TERM=rxvt-unicode-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: zfs-linux UpgradeStatus: Upgraded to bionic on 2018-08-16 (174 days ago) modified.conffile..etc.sudoers.d.zfs: [inaccessible: [Errno 13] Permission denied: '/etc/sudoers.d/zfs'] To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1814983/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp