Debugging by voodoo dolls: I made a wild guess that stumbling over
cached objects was getting in the way. I dropped caches with:

echo 3 > /proc/sys/vm/drop_caches

and immediately the read iops to the pool skyrocketed and the ripgrep
output sure looks like it's going WAY faster:

                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G     75      0   403K  
    0
  mirror                                      588G   340G     35      0   211K  
    0
    sdj                                          -      -     23      0   128K  
    0
    sdk                                          -      -     11      0  83.7K  
    0
  mirror                                      588G   340G     39      0   191K  
    0
    sdl                                          -      -     16      0  79.7K  
    0
    sdm                                          -      -     22      0   112K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T      5      0  47.8K  
    0
  mirror                                      443G  2.29T      1      0  15.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      0  7.97K  
    0
  mirror                                      443G  2.29T      0      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0      0      0  
    0
  mirror                                      443G  2.29T      2      0  23.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      1      0  15.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G      3      0  16.4K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G     67      0   375K  
    0
  mirror                                      588G   340G     43      0   255K  
    0
    sdj                                          -      -     25      0   147K  
    0
    sdk                                          -      -     17      0   108K  
    0
  mirror                                      588G   340G     23      0   120K  
    0
    sdl                                          -      -      6      0  31.9K  
    0
    sdm                                          -      -     16      0  87.7K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T      3      0  15.9K  
    0
  mirror                                      443G  2.29T      0      0  3.99K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      0  3.99K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      0      0  
    0
  mirror                                      443G  2.29T      1      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      1      0  7.97K  
    0
  mirror                                      443G  2.29T      0      0  3.99K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      0  3.99K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      0      0  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G      1      0  4.98K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G     54      0   279K  
    0
  mirror                                      588G   340G     30      0   144K  
    0
    sdj                                          -      -     19      0  87.7K  
    0
    sdk                                          -      -     10      0  55.8K  
    0
  mirror                                      588G   340G     23      0   136K  
    0
    sdl                                          -      -      6      0  59.8K  
    0
    sdm                                          -      -     16      0  75.8K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T      2      0  12.0K  
    0
  mirror                                      443G  2.29T      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      0      0  
    0
  mirror                                      443G  2.29T      2      0  12.0K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      0  3.99K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      1      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0      0      0  
    0
  mirror                                      443G  2.29T      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      0      0  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      0      0  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G      1      0  11.5K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  15.6K      0  98.7M  
    0
  mirror                                      588G   340G  7.75K      0  50.0M  
    0
    sdj                                          -      -  3.98K      0  24.8M  
    0
    sdk                                          -      -  3.77K      0  25.1M  
    0
  mirror                                      588G   340G  7.88K      0  48.8M  
    0
    sdl                                          -      -  3.92K      0  23.8M  
    0
    sdm                                          -      -  3.96K      0  25.0M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    418      0  3.38M  
    0
  mirror                                      443G  2.29T    144      0   857K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     52      0   363K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     53      0   291K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     37      0   203K  
    0
  mirror                                      443G  2.29T    100      0  1.15M  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     27      0   391K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     52      0   498K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     19      0   291K  
    0
  mirror                                      443G  2.29T    173      0  1.39M  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -    103      0   713K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     27      0   199K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     41      0   514K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G     90      0   469K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  15.7K      0   110M  
    0
  mirror                                      588G   340G  8.01K      0  57.7M  
    0
    sdj                                          -      -  3.98K      0  29.3M  
    0
    sdk                                          -      -  4.02K      0  28.4M  
    0
  mirror                                      588G   340G  7.64K      0  52.6M  
    0
    sdl                                          -      -  3.82K      0  26.1M  
    0
    sdm                                          -      -  3.82K      0  26.5M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    236      0  1.12M  
    0
  mirror                                      443G  2.29T     93      0   426K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     32      0   139K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     31      0   151K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     28      0   135K  
    0
  mirror                                      443G  2.29T     44      0   235K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     11      0  87.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      8      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     23      0   108K  
    0
  mirror                                      443G  2.29T     97      0   490K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     56      0   315K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     27      0   112K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     12      0  63.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    214      0  1.02M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G      0      0      0  
    0
  mirror                                      588G   340G      0      0      0  
    0
    sdj                                          -      -      0      0      0  
    0
    sdk                                          -      -      0      0      0  
    0
  mirror                                      588G   340G      0      0      0  
    0
    sdl                                          -      -      0      0      0  
    0
    sdm                                          -      -      0      0      0  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    117      0   498K  
    0
  mirror                                      443G  2.29T     50      0   215K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     15      0  67.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     14      0  63.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     19      0  83.7K  
    0
  mirror                                      443G  2.29T     26      0   116K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      9      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      8      0  35.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      7      0  31.9K  
    0
  mirror                                      443G  2.29T     39      0   167K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     10      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     17      0  71.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     10      0  47.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    205      0  1.04M  
7.97K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G      0      0      0  
    0
  mirror                                      588G   340G      0      0      0  
    0
    sdj                                          -      -      0      0      0  
    0
    sdk                                          -      -      0      0      0  
    0
  mirror                                      588G   340G      0      0      0  
    0
    sdl                                          -      -      0      0      0  
    0
    sdm                                          -      -      0      0      0  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    127      0   558K  
    0
  mirror                                      443G  2.29T     51      0   231K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     16      0  71.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      9      0  51.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     24      0   108K  
    0
  mirror                                      443G  2.29T     35      0   159K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     10      0  43.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      6      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     17      0  83.7K  
    0
  mirror                                      443G  2.29T     39      0   167K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     18      0  79.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     11      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      8      0  39.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    202      0   989K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  12.6K      0  90.3M  
    0
  mirror                                      588G   340G  6.46K      0  46.5M  
    0
    sdj                                          -      -  3.25K      0  23.6M  
    0
    sdk                                          -      -  3.21K      0  22.9M  
    0
  mirror                                      588G   340G  6.15K      0  43.8M  
    0
    sdl                                          -      -  3.07K      0  22.0M  
    0
    sdm                                          -      -  3.08K      0  21.9M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    129      0   550K  
    0
  mirror                                      443G  2.29T     47      0   199K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     18      0  83.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     15      0  63.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     12      0  51.8K  
    0
  mirror                                      443G  2.29T     28      0   131K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      3      0  15.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     11      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     12      0  67.7K  
    0
  mirror                                      443G  2.29T     52      0   219K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     25      0   112K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     15      0  63.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     10      0  43.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    146      0   715K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  7.97K      0  54.7M  
    0
  mirror                                      588G   340G  3.77K      0  26.1M  
    0
    sdj                                          -      -  1.92K      0  13.1M  
    0
    sdk                                          -      -  1.85K      0  13.0M  
    0
  mirror                                      588G   340G  4.20K      0  28.6M  
    0
    sdl                                          -      -  2.14K      0  13.7M  
    0
    sdm                                          -      -  2.06K      0  14.9M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    112      0   482K  
    0
  mirror                                      443G  2.29T     36      0   155K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     11      0  51.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     10      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     13      0  59.7K  
    0
  mirror                                      443G  2.29T     25      0   115K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      9      0  39.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      8      0  39.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      6      0  35.8K  
    0
  mirror                                      443G  2.29T     49      0   211K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     26      0   111K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     14      0  63.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      7      0  35.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    170      1   880K  
15.9K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  8.45K      0  59.9M  
    0
  mirror                                      588G   340G  4.26K      0  30.4M  
    0
    sdj                                          -      -  2.13K      0  15.5M  
    0
    sdk                                          -      -  2.13K      0  15.0M  
    0
  mirror                                      588G   340G  4.19K      0  29.4M  
    0
    sdl                                          -      -  2.16K      0  15.0M  
    0
    sdm                                          -      -  2.03K      0  14.4M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     87      0   394K  
    0
  mirror                                      443G  2.29T     27      0   128K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     10      0  55.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      7      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      8      0  39.8K  
    0
  mirror                                      443G  2.29T     30      0   139K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      4      0  23.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     16      0  67.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      8      0  47.8K  
    0
  mirror                                      443G  2.29T     28      0   128K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      6      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     11      0  55.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      9      0  43.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    174      0   839K  
7.97K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  11.3K      0  82.7M  
    0
  mirror                                      588G   340G  5.56K      0  39.6M  
    0
    sdj                                          -      -  2.79K      0  19.7M  
    0
    sdk                                          -      -  2.77K      0  19.9M  
    0
  mirror                                      588G   340G  5.71K      0  43.1M  
    0
    sdl                                          -      -  2.92K      0  22.1M  
    0
    sdm                                          -      -  2.79K      0  21.1M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    584    495  2.83M  
2.75M
  mirror                                      443G  2.29T    178    172   893K  
 981K
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     59     60   347K  
 327K
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     55     61   243K  
 327K
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     62     49   303K  
 327K
  mirror                                      443G  2.29T    183    172   849K  
 981K
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     69     58   299K  
 327K
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     83     59   411K  
 327K
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     29     53   140K  
 327K
  mirror                                      443G  2.29T    222    150  1.13M  
 849K
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     60     50   331K  
 283K
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     85     51   387K  
 283K
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     75     47   435K  
 283K
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    112      0   598K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  3.50K      0  24.4M  
    0
  mirror                                      588G   340G  1.74K      0  12.1M  
    0
    sdj                                          -      -    898      0  6.03M  
    0
    sdk                                          -      -    880      0  6.08M  
    0
  mirror                                      588G   340G  1.76K      0  12.3M  
    0
    sdl                                          -      -    895      0  6.30M  
    0
    sdm                                          -      -    906      0  5.98M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     74      0   323K  
    0
  mirror                                      443G  2.29T     24      0   108K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      4      0  19.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      8      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     10      0  47.8K  
    0
  mirror                                      443G  2.29T     19      0  87.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      5      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      6      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      6      0  31.9K  
    0
  mirror                                      443G  2.29T     29      0   128K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     16      0  71.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      7      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      4      0  23.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    143      1   815K  
15.9K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  2.87K      0  23.3M  
    0
  mirror                                      588G   340G  1.38K      0  12.3M  
    0
    sdj                                          -      -    673      0  5.89M  
    0
    sdk                                          -      -    735      0  6.40M  
    0
  mirror                                      588G   340G  1.49K      0  11.0M  
    0
    sdl                                          -      -    758      0  5.96M  
    0
    sdm                                          -      -    767      0  5.09M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    534      0  2.35M  
    0
  mirror                                      443G  2.29T    141      0   733K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     57      0   263K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     45      0   259K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     37      0   211K  
    0
  mirror                                      443G  2.29T    156      0   653K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     48      0   207K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     78      0   323K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     28      0   123K  
    0
  mirror                                      443G  2.29T    236      0  1023K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     93      0   410K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -    130      0   558K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     12      0  55.7K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    163      0   761K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G      4      0  19.9K  
    0
  mirror                                      588G   340G      1      0  7.96K  
    0
    sdj                                          -      -      1      0  7.96K  
    0
    sdk                                          -      -      0      0      0  
    0
  mirror                                      588G   340G      2      0  11.9K  
    0
    sdl                                          -      -      0      0  3.98K  
    0
    sdm                                          -      -      1      0  7.96K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     50      0   362K  
    0
  mirror                                      443G  2.29T     26      0   163K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      4      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     11      0  83.6K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      9      0  51.8K  
    0
  mirror                                      443G  2.29T      8      0  51.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      3      0  23.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      3      0  19.9K  
    0
  mirror                                      443G  2.29T     14      0   147K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      5      0  83.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      3      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      4      0  35.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G     14      0  72.7K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  8.64K      0  62.3M  
    0
  mirror                                      588G   340G  4.39K      0  31.7M  
    0
    sdj                                          -      -  2.22K      0  16.2M  
    0
    sdk                                          -      -  2.18K      0  15.6M  
    0
  mirror                                      588G   340G  4.25K      0  30.6M  
    0
    sdl                                          -      -  2.15K      0  15.4M  
    0
    sdm                                          -      -  2.10K      0  15.2M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    124      0   805K  
    0
  mirror                                      443G  2.29T     53      0   271K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     18      0  87.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     13      0  63.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     20      0   120K  
    0
  mirror                                      443G  2.29T     29      0   147K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     11      0  55.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      8      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      8      0  43.8K  
    0
  mirror                                      443G  2.29T     40      0   387K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     22      0   307K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      8      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      8      0  39.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    118      0   649K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  6.73K      0  44.6M  
    0
  mirror                                      588G   340G  3.20K      0  21.4M  
    0
    sdj                                          -      -  1.61K      0  10.4M  
    0
    sdk                                          -      -  1.59K      0  11.0M  
    0
  mirror                                      588G   340G  3.53K      0  23.2M  
    0
    sdl                                          -      -  1.82K      0  11.7M  
    0
    sdm                                          -      -  1.72K      0  11.5M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    199      0  1.23M  
    0
  mirror                                      443G  2.29T     55      0   335K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     17      0   144K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     16      0  83.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     20      0   108K  
    0
  mirror                                      443G  2.29T     62      0   403K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     30      0   140K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     11      0   175K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     19      0  87.7K  
    0
  mirror                                      443G  2.29T     80      0   526K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     50      0   243K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     17      0   175K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     11      0   108K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    202      0   999K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  10.8K      0  65.4M  
    0
  mirror                                      588G   340G  5.45K      0  32.7M  
    0
    sdj                                          -      -  2.82K      0  16.7M  
    0
    sdk                                          -      -  2.63K      0  15.9M  
    0
  mirror                                      588G   340G  5.38K      0  32.7M  
    0
    sdl                                          -      -  2.74K      0  16.5M  
    0
    sdm                                          -      -  2.64K      0  16.2M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    236      0  2.09M  
    0
  mirror                                      443G  2.29T     85      0   470K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     36      0   195K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     24      0   124K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     23      0   151K  
    0
  mirror                                      443G  2.29T     93      0   773K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     42      0   243K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     10      0   104K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     39      0   426K  
    0
  mirror                                      443G  2.29T     56      0   901K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     27      0   482K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      9      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     18      0   375K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    138      0   607K  
7.97K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  13.4K      0   167M  
    0
  mirror                                      588G   340G  6.94K      0  85.5M  
    0
    sdj                                          -      -  3.48K      0  43.6M  
    0
    sdk                                          -      -  3.46K      0  41.8M  
    0
  mirror                                      588G   340G  6.50K      0  81.4M  
    0
    sdl                                          -      -  3.26K      0  41.5M  
    0
    sdm                                          -      -  3.24K      0  39.9M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    112      0   502K  
    0
  mirror                                      443G  2.29T     35      0   151K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     13      0  59.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      6      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     14      0  63.7K  
    0
  mirror                                      443G  2.29T     33      0   167K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      9      0  55.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     11      0  51.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     11      0  59.8K  
    0
  mirror                                      443G  2.29T     42      0   183K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     21      0  95.6K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      6      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     13      0  59.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    246      0  1.11M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  10.5K      0   365M  
    0
  mirror                                      588G   340G  5.24K      0   184M  
    0
    sdj                                          -      -  2.56K      0  89.6M  
    0
    sdk                                          -      -  2.68K      0  94.7M  
    0
  mirror                                      588G   340G  5.29K      0   181M  
    0
    sdl                                          -      -  2.76K      0  94.5M  
    0
    sdm                                          -      -  2.52K      0  86.4M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    109      0   494K  
    0
  mirror                                      443G  2.29T     27      0   124K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      5      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     13      0  59.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      7      0  31.9K  
    0
  mirror                                      443G  2.29T     42      0   203K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     12      0  59.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     14      0  71.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     14      0  71.8K  
    0
  mirror                                      443G  2.29T     38      0   167K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     24      0   104K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      6      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      6      0  31.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    373      0  1.80M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  19.2K      0  94.0M  
    0
  mirror                                      588G   340G  9.77K      0  47.2M  
    0
    sdj                                          -      -  4.93K      0  24.0M  
    0
    sdk                                          -      -  4.84K      0  23.2M  
    0
  mirror                                      588G   340G  9.45K      0  46.8M  
    0
    sdl                                          -      -  4.75K      0  23.5M  
    0
    sdm                                          -      -  4.69K      0  23.3M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     65      0   295K  
    0
  mirror                                      443G  2.29T      7      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      2      0  15.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      1      0  7.97K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      2      0  15.9K  
    0
  mirror                                      443G  2.29T     29      0   136K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      8      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     12      0  59.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      7      0  35.9K  
    0
  mirror                                      443G  2.29T     27      0   120K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     19      0  83.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      2      0  12.0K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      4      0  23.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    162      0   711K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  12.6K      0  94.8M  
    0
  mirror                                      588G   340G  7.28K      0  54.5M  
    0
    sdj                                          -      -  3.68K      0  26.2M  
    0
    sdk                                          -      -  3.60K      0  28.2M  
    0
  mirror                                      588G   340G  5.33K      0  40.3M  
    0
    sdl                                          -      -  2.61K      0  19.8M  
    0
    sdm                                          -      -  2.72K      0  20.5M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    129    548   554K  
5.98M
  mirror                                      443G  2.29T     28    214   120K  
4.16M
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     10     66  43.9K  
1.39M
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      9     74  39.9K  
1.39M
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      7     72  35.9K  
1.39M
  mirror                                      443G  2.29T     53    181   235K  
1.05M
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     26     57   116K  
 359K
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     17     62  75.8K  
 359K
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      8     60  43.9K  
 359K
  mirror                                      443G  2.29T     46    152   199K  
 790K
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     25     48   108K  
 263K
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     11     48  51.8K  
 263K
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      8     54  39.9K  
 263K
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    272      0  1.10M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  14.3K      0   104M  
    0
  mirror                                      588G   340G  7.26K      0  54.0M  
    0
    sdj                                          -      -  3.66K      0  26.9M  
    0
    sdk                                          -      -  3.60K      0  27.2M  
    0
  mirror                                      588G   340G  7.05K      0  49.6M  
    0
    sdl                                          -      -  3.49K      0  23.8M  
    0
    sdm                                          -      -  3.56K      0  25.8M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    111      0   482K  
    0
  mirror                                      443G  2.29T     25      0   116K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      9      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      7      0  35.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      7      0  35.9K  
    0
  mirror                                      443G  2.29T     49      0   219K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     15      0  71.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     18      0  75.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     14      0  71.7K  
    0
  mirror                                      443G  2.29T     35      0   147K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     20      0  83.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      4      0  23.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      9      0  39.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    216      0  1.04M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  14.7K      0   105M  
    0
  mirror                                      588G   340G  9.21K      0  66.1M  
    0
    sdj                                          -      -  4.61K      0  33.2M  
    0
    sdk                                          -      -  4.60K      0  32.9M  
    0
  mirror                                      588G   340G  5.49K      0  39.3M  
    0
    sdl                                          -      -  2.73K      0  19.3M  
    0
    sdm                                          -      -  2.76K      0  20.1M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     98      0   438K  
    0
  mirror                                      443G  2.29T     25      0   120K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     13      0  71.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      4      0  19.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      6      0  27.9K  
    0
  mirror                                      443G  2.29T     43      0   199K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     20      0  91.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      7      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     14      0  75.7K  
    0
  mirror                                      443G  2.29T     28      0   120K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     12      0  55.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      7      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      7      0  31.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    206      0   990K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  13.8K      0  92.5M  
    0
  mirror                                      588G   340G  6.88K      0  46.9M  
    0
    sdj                                          -      -  3.57K      0  23.6M  
    0
    sdk                                          -      -  3.31K      0  23.4M  
    0
  mirror                                      588G   340G  6.91K      0  45.6M  
    0
    sdl                                          -      -  3.50K      0  23.2M  
    0
    sdm                                          -      -  3.41K      0  22.4M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    104      0   467K  
    0
  mirror                                      443G  2.29T     34      0   155K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      9      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     11      0  51.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     12      0  55.8K  
    0
  mirror                                      443G  2.29T     30      0   136K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     14      0  67.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      7      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      7      0  35.9K  
    0
  mirror                                      443G  2.29T     38      0   175K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     18      0  79.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      7      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     11      0  55.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    245      0  1.03M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  13.5K      0   127M  
    0
  mirror                                      588G   340G  6.54K      0  60.4M  
    0
    sdj                                          -      -  3.29K      0  30.6M  
    0
    sdk                                          -      -  3.25K      0  29.8M  
    0
  mirror                                      588G   340G  6.95K      0  67.1M  
    0
    sdl                                          -      -  3.50K      0  33.8M  
    0
    sdm                                          -      -  3.46K      0  33.4M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    101      0   434K  
    0
  mirror                                      443G  2.29T     41      0   171K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     17      0  75.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      9      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     13      0  55.8K  
    0
  mirror                                      443G  2.29T     23      0   108K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      9      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      6      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      6      0  31.9K  
    0
  mirror                                      443G  2.29T     35      0   155K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     13      0  59.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      7      0  31.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     13      0  63.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    265      0  1.16M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  14.7K      0  94.1M  
    0
  mirror                                      588G   340G  7.37K      0  46.2M  
    0
    sdj                                          -      -  3.73K      0  22.9M  
    0
    sdk                                          -      -  3.64K      0  23.2M  
    0
  mirror                                      588G   340G  7.27K      0  47.9M  
    0
    sdl                                          -      -  3.75K      0  24.8M  
    0
    sdm                                          -      -  3.52K      0  23.1M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    107      0   466K  
    0
  mirror                                      443G  2.29T     48      0   211K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      9      0  39.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     17      0  79.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     20      0  91.7K  
    0
  mirror                                      443G  2.29T     29      0   124K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     10      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     11      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      6      0  27.9K  
    0
  mirror                                      443G  2.29T     28      0   132K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      9      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     12      0  55.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      5      0  31.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    137      0   644K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  21.1K      0   129M  
    0
  mirror                                      588G   340G  10.7K      0  64.1M  
    0
    sdj                                          -      -  5.41K      0  32.1M  
    0
    sdk                                          -      -  5.25K      0  32.0M  
    0
  mirror                                      588G   340G  10.4K      0  64.6M  
    0
    sdl                                          -      -  5.32K      0  33.4M  
    0
    sdm                                          -      -  5.09K      0  31.2M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    372      0  4.05M  
    0
  mirror                                      443G  2.29T    146      0  1019K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     47      0   263K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     51      0   291K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     46      0   466K  
    0
  mirror                                      443G  2.29T     79      0  1.06M  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     25      0   263K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -     18      0   573K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     34      0   251K  
    0
  mirror                                      443G  2.29T    146      0  2.00M  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     50      0  1.38M  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     64      0   279K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     30      0   350K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    164      0   737K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  20.2K      0   251M  
    0
  mirror                                      588G   340G  10.6K      0   131M  
    0
    sdj                                          -      -  5.19K      0  63.8M  
    0
    sdk                                          -      -  5.39K      0  66.8M  
    0
  mirror                                      588G   340G  9.63K      0   121M  
    0
    sdl                                          -      -  4.79K      0  59.4M  
    0
    sdm                                          -      -  4.84K      0  61.5M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    102      0   450K  
    0
  mirror                                      443G  2.29T     53      0   247K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     12      0  51.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -     13      0  59.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     26      0   135K  
    0
  mirror                                      443G  2.29T     31      0   127K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     14      0  59.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      8      0  35.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      7      0  31.8K  
    0
  mirror                                      443G  2.29T     16      0  75.6K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      2      0  11.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      5      0  23.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      7      0  39.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    143      0   527K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  22.3K      0   275M  
    0
  mirror                                      588G   340G  11.2K      0   133M  
    0
    sdj                                          -      -  5.64K      0  66.6M  
    0
    sdk                                          -      -  5.56K      0  66.6M  
    0
  mirror                                      588G   340G  11.1K      0   142M  
    0
    sdl                                          -      -  5.62K      0  70.2M  
    0
    sdm                                          -      -  5.49K      0  71.5M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     82      0   382K  
    0
  mirror                                      443G  2.29T     25      0   123K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      9      0  47.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      6      0  35.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      8      0  39.8K  
    0
  mirror                                      443G  2.29T     25      0   131K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     11      0  55.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      6      0  35.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      6      0  39.8K  
    0
  mirror                                      443G  2.29T     30      0   127K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      7      0  35.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     11      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     10      0  43.8K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    172      2   731K  
19.9K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  27.2K      0   178M  
    0
  mirror                                      588G   340G  13.8K      0  90.7M  
    0
    sdj                                          -      -  6.91K      0  45.4M  
    0
    sdk                                          -      -  6.87K      0  45.3M  
    0
  mirror                                      588G   340G  13.4K      0  87.4M  
    0
    sdl                                          -      -  6.74K      0  43.9M  
    0
    sdm                                          -      -  6.68K      0  43.6M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T    111      0   669K  
    0
  mirror                                      443G  2.29T     31      0   171K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     11      0  79.6K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      9      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      9      0  43.8K  
    0
  mirror                                      443G  2.29T     54      0   338K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     14      0  63.7K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      7      0  31.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     31      0   243K  
    0
  mirror                                      443G  2.29T     24      0   159K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      4      0  23.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     10      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      8      0  91.5K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    167      2   612K  
19.9K
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  30.7K      0   220M  
    0
  mirror                                      588G   340G  14.7K      0   105M  
    0
    sdj                                          -      -  7.41K      0  53.7M  
    0
    sdk                                          -      -  7.29K      0  51.7M  
    0
  mirror                                      588G   340G  15.9K      0   114M  
    0
    sdl                                          -      -  8.04K      0  57.4M  
    0
    sdm                                          -      -  7.91K      0  57.0M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     91    387   386K  
6.15M
  mirror                                      443G  2.29T     30    139   131K  
1.03M
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -     11     46  51.8K  
 350K
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      9     47  43.8K  
 350K
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      8     44  35.8K  
 350K
  mirror                                      443G  2.29T     28    109   127K  
 860K
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      6     35  27.9K  
 287K
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      6     36  31.9K  
 287K
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -     14     36  67.7K  
 287K
  mirror                                      443G  2.29T     31    138   127K  
4.28M
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -     15     47  63.7K  
1.43M
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      4     43  19.9K  
1.43M
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -     10     46  43.8K  
1.43M
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    126      0   560K  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
                                               capacity     operations     
bandwidth 
pool                                         alloc   free   read  write   read  
write
-------------------------------------------  -----  -----  -----  -----  -----  
-----
fst                                          1.15T   679G  25.0K      0   206M  
    0
  mirror                                      588G   340G  12.3K      0   103M  
    0
    sdj                                          -      -  6.21K      0  51.9M  
    0
    sdk                                          -      -  6.13K      0  50.7M  
    0
  mirror                                      588G   340G  12.6K      0   103M  
    0
    sdl                                          -      -  6.31K      0  51.6M  
    0
    sdm                                          -      -  6.30K      0  51.5M  
    0
-------------------------------------------  -----  -----  -----  -----  -----  
-----
srv                                          1.30T  6.86T     83      0   366K  
    0
  mirror                                      443G  2.29T     26      0   123K  
    0
    ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      9      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      5      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -     10      0  51.8K  
    0
  mirror                                      443G  2.29T     30      0   135K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -     10      0  47.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      9      0  43.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      9      0  43.8K  
    0
  mirror                                      443G  2.29T     25      0   108K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      5      0  27.9K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -     12      0  51.8K  
    0
    ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      6      0  27.9K  
    0
logs                                             -      -      -      -      -  
    -
  nvme0n1p1                                   900K  19.9G      0      0      0  
    0
cache                                            -      -      -      -      -  
    -
  nvme0n1p2                                   334G   764G    198      2   770K  
15.9K
-------------------------------------------  -----  -----  -----  -----  -----  
-----

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to zfs-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1814983

Title:
  zfs poor sustained read performance from ssd pool

Status in zfs-linux package in Ubuntu:
  New

Bug description:
  Hello,

  I'm seeing substantially slower read performance from an ssd pool than
  I expected.

  I have two pools on this computer; one ('fst') is four sata ssds, the
  other ('srv') is nine spinning metal drives.

  With a long-running ripgrep process on the fst pool, performance
  started out really good and grew to astonishingly good (iirc ~30kiops,
  as measured by zpool iostat -v 1). However after a few hours the
  performance has dropped to 30-40 iops. top reports an arc_reclaim and
  many arc_prune processes to be consuming most of the CPU time.

  I've included a screenshot of top, some output from zpool iostat -v 1,
  and arc_summary, with "===" to indicate the start of the next
  command's output:

  ===
  top (memory in gigabytes):

  top - 16:27:53 up 70 days, 16:03,  3 users,  load average: 35.67, 35.81, 35.58
  Tasks: 809 total,  19 running, 612 sleeping,   0 stopped,   0 zombie
  %Cpu(s):  0.0 us, 58.1 sy,  0.0 ni, 39.2 id,  2.6 wa,  0.0 hi,  0.0 si,  0.0 
st
  GiB Mem :  125.805 total,    0.620 free,   96.942 used,   28.243 buff/cache
  GiB Swap:    5.694 total,    5.688 free,    0.006 used.   27.840 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND   
                                      
   1523 root      20   0    0.0m   0.0m   0.0m R 100.0  0.0 290:52.26 
arc_reclaim                                     
   4484 root      20   0    0.0m   0.0m   0.0m R  56.2  0.0   1:18.79 arc_prune 
                                      
   6225 root      20   0    0.0m   0.0m   0.0m R  56.2  0.0   1:11.92 arc_prune 
                                      
   7601 root      20   0    0.0m   0.0m   0.0m S  56.2  0.0   2:50.25 arc_prune 
                                      
  30891 root      20   0    0.0m   0.0m   0.0m S  56.2  0.0   1:33.08 arc_prune 
                                      
   3057 root      20   0    0.0m   0.0m   0.0m S  55.9  0.0   9:00.95 arc_prune 
                                      
   3259 root      20   0    0.0m   0.0m   0.0m R  55.9  0.0   3:16.84 arc_prune 
                                      
  24008 root      20   0    0.0m   0.0m   0.0m S  55.9  0.0   1:55.71 arc_prune 
                                      
   1285 root      20   0    0.0m   0.0m   0.0m R  55.6  0.0   3:20.52 arc_prune 
                                      
   5345 root      20   0    0.0m   0.0m   0.0m R  55.6  0.0   1:15.99 arc_prune 
                                      
  30121 root      20   0    0.0m   0.0m   0.0m S  55.6  0.0   1:35.50 arc_prune 
                                      
  31192 root      20   0    0.0m   0.0m   0.0m S  55.6  0.0   6:17.16 arc_prune 
                                      
  32287 root      20   0    0.0m   0.0m   0.0m S  55.6  0.0   1:28.02 arc_prune 
                                      
  32625 root      20   0    0.0m   0.0m   0.0m R  55.6  0.0   1:27.34 arc_prune 
                                      
  22572 root      20   0    0.0m   0.0m   0.0m S  55.3  0.0  10:02.92 arc_prune 
                                      
  31989 root      20   0    0.0m   0.0m   0.0m R  55.3  0.0   1:28.03 arc_prune 
                                      
   3353 root      20   0    0.0m   0.0m   0.0m R  54.9  0.0   8:58.81 arc_prune 
                                      
  10252 root      20   0    0.0m   0.0m   0.0m R  54.9  0.0   2:36.37 arc_prune 
                                      
   1522 root      20   0    0.0m   0.0m   0.0m S  53.9  0.0 158:42.45 arc_prune 
                                      
   3694 root      20   0    0.0m   0.0m   0.0m R  53.9  0.0   1:20.79 arc_prune 
                                      
  13394 root      20   0    0.0m   0.0m   0.0m R  53.9  0.0  10:35.78 arc_prune 
                                      
  24592 root      20   0    0.0m   0.0m   0.0m R  53.9  0.0   1:54.19 arc_prune 
                                      
  25859 root      20   0    0.0m   0.0m   0.0m S  53.9  0.0   1:51.71 arc_prune 
                                      
   8194 root      20   0    0.0m   0.0m   0.0m S  53.6  0.0   0:54.51 arc_prune 
                                      
  18472 root      20   0    0.0m   0.0m   0.0m R  53.6  0.0   2:08.73 arc_prune 
                                      
  29525 root      20   0    0.0m   0.0m   0.0m R  53.6  0.0   1:35.81 arc_prune 
                                      
  32291 root      20   0    0.0m   0.0m   0.0m S  53.6  0.0   1:28.00 arc_prune 
                                      
   3156 root      20   0    0.0m   0.0m   0.0m R  53.3  0.0   3:17.68 arc_prune 
                                      
   6224 root      20   0    0.0m   0.0m   0.0m S  53.3  0.0   1:11.80 arc_prune 
                                      
   9788 root      20   0    0.0m   0.0m   0.0m S  53.3  0.0   0:46.00 arc_prune 
                                      
  10341 root      20   0    0.0m   0.0m   0.0m R  53.3  0.0   2:36.23 arc_prune 
                                      
  11881 root      20   0    0.0m   0.0m   0.0m S  53.0  0.0   2:31.57 arc_prune 
                                      
  24030 root      20   0    0.0m   0.0m   0.0m R  52.6  0.0   1:55.44 arc_prune 
                                      

  ===
  zpool iostat -v 1  output (for a while):
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G     68      0   
650K      0
    mirror                                      588G   340G     31      0   
331K      0
      sdj                                          -      -     20      0   
179K      0
      sdk                                          -      -     10      0   
152K      0
    mirror                                      588G   340G     36      0   
319K      0
      sdl                                          -      -     17      0   
132K      0
      sdm                                          -      -     18      0   
187K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      2     35   
187K   144K
    mirror                                      443G  2.29T      0      0  
63.8K      0
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      0  
63.8K      0
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      0      
0      0
    mirror                                      443G  2.29T      0     17      
0  71.8K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      5      
0  23.9K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      5      
0  23.9K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0      5      
0  23.9K
    mirror                                      443G  2.29T      1     17   
124K  71.8K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      1      5   
124K  23.9K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      5      
0  23.9K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      5      
0  23.9K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G    110      0  
1.07M      0
    mirror                                      588G   340G     59      0   
634K      0
      sdj                                          -      -     28      0   
303K      0
      sdk                                          -      -     30      0   
331K      0
    mirror                                      588G   340G     50      0   
459K      0
      sdl                                          -      -     28      0   
303K      0
      sdm                                          -      -     21      0   
155K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      2    229   
183K  1.00M
    mirror                                      443G  2.29T      2     73   
183K   335K
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      2     24   
183K   112K
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0     24      
0   112K
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0     23      
0   112K
    mirror                                      443G  2.29T      0     77      
0   347K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0     25      
0   116K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0     25      
0   116K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0     25      
0   116K
    mirror                                      443G  2.29T      0     77      
0   347K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      0     25      
0   116K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0     25      
0   116K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0     25      
0   116K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G     29      0   
403K      0
    mirror                                      588G   340G     12      0   
171K      0
      sdj                                          -      -      7      0  
79.7K      0
      sdk                                          -      -      4      0  
91.7K      0
    mirror                                      588G   340G     16      0   
231K      0
      sdl                                          -      -      6      0   
128K      0
      sdm                                          -      -      9      0   
104K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      0     66  
63.8K   359K
    mirror                                      443G  2.29T      0     21      
0   120K
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      6      
0  39.9K
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      7      
0  39.9K
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      6      
0  39.9K
    mirror                                      443G  2.29T      0     21      
0   120K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      7      
0  39.9K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      6      
0  39.9K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0      6      
0  39.9K
    mirror                                      443G  2.29T      0     22  
63.8K   120K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      0      7  
63.8K  39.9K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      6      
0  39.9K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      7      
0  39.9K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G     97      0   
797K      0
    mirror                                      588G   340G     58      0   
474K      0
      sdj                                          -      -     27      0   
263K      0
      sdk                                          -      -     30      0   
211K      0
    mirror                                      588G   340G     38      0   
323K      0
      sdl                                          -      -     23      0   
203K      0
      sdm                                          -      -     14      0   
120K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      2    176   
187K   789K
    mirror                                      443G  2.29T      0     58  
59.8K   263K
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0     19  
59.8K  87.7K
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0     18      
0  87.7K
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0     19      
0  87.7K
    mirror                                      443G  2.29T      0     59      
0   263K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0     19      
0  87.7K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0     19      
0  87.7K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0     19      
0  87.7K
    mirror                                      443G  2.29T      1     57   
128K   263K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      1     18   
128K  87.7K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0     19      
0  87.7K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0     18      
0  87.7K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G     70      0   
426K      0
    mirror                                      588G   340G     38      0   
263K      0
      sdj                                          -      -     21      0   
128K      0
      sdk                                          -      -     16      0   
135K      0
    mirror                                      588G   340G     31      0   
163K      0
      sdl                                          -      -     10      0  
67.7K      0
      sdm                                          -      -     20      0  
95.6K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      1     46   
116K  2.36M
    mirror                                      443G  2.29T      0      0  
59.8K      0
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      0  
59.8K      0
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      0      
0      0
    mirror                                      443G  2.29T      0     37      
0  2.31M
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0     12      
0   789K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0     11      
0   789K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0     12      
0   789K
    mirror                                      443G  2.29T      0      8  
55.8K  47.8K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      0      2  
55.8K  15.9K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      2      
0  15.9K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      2      
0  15.9K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G    108      0   
614K      0
    mirror                                      588G   340G     50      0   
299K      0
      sdj                                          -      -     32      0   
203K      0
      sdk                                          -      -     17      0  
95.6K      0
    mirror                                      588G   340G     57      0   
315K      0
      sdl                                          -      -     30      0   
155K      0
      sdm                                          -      -     26      0   
159K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      2     68   
191K   311K
    mirror                                      443G  2.29T      0      8      
0  47.8K
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      2      
0  15.9K
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      2      
0  15.9K
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      2      
0  15.9K
    mirror                                      443G  2.29T      0     29      
0   132K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      9      
0  43.8K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      9      
0  43.8K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0      9      
0  43.8K
    mirror                                      443G  2.29T      2     29   
191K   132K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      2      9   
191K  43.8K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      9      
0  43.8K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      9      
0  43.8K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G     66      0   
379K      0
    mirror                                      588G   340G     26      0   
144K      0
      sdj                                          -      -     12      0  
63.8K      0
      sdk                                          -      -     13      0  
79.7K      0
    mirror                                      588G   340G     39      0   
235K      0
      sdl                                          -      -     19      0   
120K      0
      sdm                                          -      -     19      0   
116K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      2    166   
183K   754K
    mirror                                      443G  2.29T      0     55      
0   251K
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0     18      
0  83.7K
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0     17      
0  83.7K
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0     18      
0  83.7K
    mirror                                      443G  2.29T      0     54      
0   251K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0     17      
0  83.7K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0     18      
0  83.7K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0     17      
0  83.7K
    mirror                                      443G  2.29T      2     55   
183K   251K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      2     18   
183K  83.7K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0     17      
0  83.7K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0     18      
0  83.7K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G    126      0   
698K      0
    mirror                                      588G   340G     64      0   
335K      0
      sdj                                          -      -     37      0   
195K      0
      sdk                                          -      -     26      0   
140K      0
    mirror                                      588G   340G     61      0   
363K      0
      sdl                                          -      -     34      0   
207K      0
      sdm                                          -      -     26      0   
155K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      3    274   
239K  1.23M
    mirror                                      443G  2.29T      1     91   
120K   418K
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      1     30   
120K   139K
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0     29      
0   139K
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0     30      
0   139K
    mirror                                      443G  2.29T      0     91      
0   418K
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0     30      
0   139K
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0     29      
0   139K
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0     30      
0   139K
    mirror                                      443G  2.29T      1     91   
120K   418K
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      1     30   
120K   139K
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0     30      
0   139K
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0     29      
0   139K
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
                                                 capacity     operations     
bandwidth 
  pool                                         alloc   free   read  write   
read  write
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  fst                                          1.15T   679G     70      0   
442K      0
    mirror                                      588G   340G     36      0   
215K      0
      sdj                                          -      -     18      0  
95.6K      0
      sdk                                          -      -     17      0   
119K      0
    mirror                                      588G   340G     33      0   
227K      0
      sdl                                          -      -     15      0   
123K      0
      sdm                                          -      -     17      0   
104K      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----
  srv                                          1.30T  6.86T      2      0   
187K      0
    mirror                                      443G  2.29T      0      0  
63.7K      0
      ata-HGST_HUS724030ALA640_PN2234P8KTWJYY      -      -      0      0  
63.7K      0
      ata-HGST_HUS724030ALA640_PN2234P9G620TW      -      -      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G66E2U      -      -      0      0      
0      0
    mirror                                      443G  2.29T      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G69TKU      -      -      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G69TXU      -      -      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G69U2U      -      -      0      0      
0      0
    mirror                                      443G  2.29T      1      0   
123K      0
      ata-HGST_HUS724030ALA640_PN2234P9G6EBUU      -      -      1      0   
123K      0
      ata-HGST_HUS724030ALA640_PN2234P9G6ESAU      -      -      0      0      
0      0
      ata-HGST_HUS724030ALA640_PN2234P9G6G70U      -      -      0      0      
0      0
  logs                                             -      -      -      -      
-      -
    nvme0n1p1                                   900K  19.9G      0      0      
0      0
  cache                                            -      -      -      -      
-      -
    nvme0n1p2                                   334G   764G      0      0      
0      0
  -------------------------------------------  -----  -----  -----  -----  
-----  -----

  
  ===
  arc_summary

  ------------------------------------------------------------------------
  ZFS Subsystem Report                          Wed Feb 06 16:34:09 2019
  ARC Summary: (HEALTHY)
        Memory Throttle Count:                  0

  ARC Misc:
        Deleted:                                142.80m
        Mutex Misses:                           14.89m
        Evict Skips:                            21.87b

  ARC Size:                             86.88%  54.65   GiB
        Target Size: (Adaptive)         78.61%  49.45   GiB
        Min Size (Hard Limit):          6.25%   3.93    GiB
        Max Size (High Water):          16:1    62.90   GiB

  ARC Size Breakdown:
        Recently Used Cache Size:       25.05%  5.55    GiB
        Frequently Used Cache Size:     74.95%  16.62   GiB

  ARC Hash Breakdown:
        Elements Max:                           7.46m
        Elements Current:               60.33%  4.50m
        Collisions:                             43.79m
        Chain Max:                              8
        Chains:                                 504.81k

  ARC Total accesses:                                   2.51b
        Cache Hit Ratio:                95.24%  2.39b
        Cache Miss Ratio:               4.76%   119.32m
        Actual Hit Ratio:               92.99%  2.33b

        Data Demand Efficiency:         94.16%  486.53m
        Data Prefetch Efficiency:       86.47%  29.68m

        CACHE HITS BY CACHE LIST:
          Anonymously Used:             2.20%   52.51m
          Most Recently Used:           37.35%  891.38m
          Most Frequently Used:         60.29%  1.44b
          Most Recently Used Ghost:     0.10%   2.32m
          Most Frequently Used Ghost:   0.06%   1.45m

        CACHE HITS BY DATA TYPE:
          Demand Data:                  19.20%  458.13m
          Prefetch Data:                1.08%   25.66m
          Demand Metadata:              78.22%  1.87b
          Prefetch Metadata:            1.51%   36.00m

        CACHE MISSES BY DATA TYPE:
          Demand Data:                  23.80%  28.40m
          Prefetch Data:                3.37%   4.02m
          Demand Metadata:              66.03%  78.79m
          Prefetch Metadata:            6.80%   8.12m

  L2 ARC Summary: (HEALTHY)
        Low Memory Aborts:                      233
        Free on Write:                          27.52k
        R/W Clashes:                            0
        Bad Checksums:                          0
        IO Errors:                              0

  L2 ARC Size: (Adaptive)                               364.94  GiB
        Compressed:                     91.59%  334.23  GiB
        Header Size:                    0.08%   307.98  MiB

  L2 ARC Breakdown:                             119.32m
        Hit Ratio:                      1.42%   1.69m
        Miss Ratio:                     98.58%  117.63m
        Feeds:                                  6.01m

  L2 ARC Writes:
        Writes Sent:                    100.00% 279.55k

  DMU Prefetch Efficiency:                                      1.89b
        Hit Ratio:                      2.24%   42.49m
        Miss Ratio:                     97.76%  1.85b


  ZFS Tunable:
        dbuf_cache_hiwater_pct                            10
        dbuf_cache_lowater_pct                            10
        dbuf_cache_max_bytes                              104857600
        dbuf_cache_max_shift                              5
        dmu_object_alloc_chunk_shift                      7
        ignore_hole_birth                                 1
        l2arc_feed_again                                  1
        l2arc_feed_min_ms                                 200
        l2arc_feed_secs                                   1
        l2arc_headroom                                    2
        l2arc_headroom_boost                              200
        l2arc_noprefetch                                  1
        l2arc_norw                                        0
        l2arc_write_boost                                 8388608
        l2arc_write_max                                   8388608
        metaslab_aliquot                                  524288
        metaslab_bias_enabled                             1
        metaslab_debug_load                               0
        metaslab_debug_unload                             0
        metaslab_fragmentation_factor_enabled             1
        metaslab_lba_weighting_enabled                    1
        metaslab_preload_enabled                          1
        metaslabs_per_vdev                                200
        send_holes_without_birth_time                     1
        spa_asize_inflation                               24
        spa_config_path                                   /etc/zfs/zpool.cache
        spa_load_verify_data                              1
        spa_load_verify_maxinflight                       10000
        spa_load_verify_metadata                          1
        spa_slop_shift                                    5
        zfetch_array_rd_sz                                1048576
        zfetch_max_distance                               8388608
        zfetch_max_streams                                8
        zfetch_min_sec_reap                               2
        zfs_abd_scatter_enabled                           1
        zfs_abd_scatter_max_order                         10
        zfs_admin_snapshot                                1
        zfs_arc_average_blocksize                         8192
        zfs_arc_dnode_limit                               0
        zfs_arc_dnode_limit_percent                       10
        zfs_arc_dnode_reduce_percent                      10
        zfs_arc_grow_retry                                0
        zfs_arc_lotsfree_percent                          10
        zfs_arc_max                                       0
        zfs_arc_meta_adjust_restarts                      4096
        zfs_arc_meta_limit                                0
        zfs_arc_meta_limit_percent                        75
        zfs_arc_meta_min                                  0
        zfs_arc_meta_prune                                10000
        zfs_arc_meta_strategy                             1
        zfs_arc_min                                       0
        zfs_arc_min_prefetch_lifespan                     0
        zfs_arc_p_aggressive_disable                      1
        zfs_arc_p_dampener_disable                        1
        zfs_arc_p_min_shift                               0
        zfs_arc_pc_percent                                0
        zfs_arc_shrink_shift                              0
        zfs_arc_sys_free                                  0
        zfs_autoimport_disable                            1
        zfs_compressed_arc_enabled                        1
        zfs_dbgmsg_enable                                 0
        zfs_dbgmsg_maxsize                                4194304
        zfs_dbuf_state_index                              0
        zfs_deadman_checktime_ms                          5000
        zfs_deadman_enabled                               1
        zfs_deadman_synctime_ms                           1000000
        zfs_dedup_prefetch                                0
        zfs_delay_min_dirty_percent                       60
        zfs_delay_scale                                   500000
        zfs_delete_blocks                                 20480
        zfs_dirty_data_max                                4294967296
        zfs_dirty_data_max_max                            4294967296
        zfs_dirty_data_max_max_percent                    25
        zfs_dirty_data_max_percent                        10
        zfs_dirty_data_sync                               67108864
        zfs_dmu_offset_next_sync                          0
        zfs_expire_snapshot                               300
        zfs_flags                                         0
        zfs_free_bpobj_enabled                            1
        zfs_free_leak_on_eio                              0
        zfs_free_max_blocks                               100000
        zfs_free_min_time_ms                              1000
        zfs_immediate_write_sz                            32768
        zfs_max_recordsize                                1048576
        zfs_mdcomp_disable                                0
        zfs_metaslab_fragmentation_threshold              70
        zfs_metaslab_segment_weight_enabled               1
        zfs_metaslab_switch_threshold                     2
        zfs_mg_fragmentation_threshold                    85
        zfs_mg_noalloc_threshold                          0
        zfs_multihost_fail_intervals                      5
        zfs_multihost_history                             0
        zfs_multihost_import_intervals                    10
        zfs_multihost_interval                            1000
        zfs_multilist_num_sublists                        0
        zfs_no_scrub_io                                   0
        zfs_no_scrub_prefetch                             0
        zfs_nocacheflush                                  0
        zfs_nopwrite_enabled                              1
        zfs_object_mutex_size                             64
        zfs_pd_bytes_max                                  52428800
        zfs_per_txg_dirty_frees_percent                   30
        zfs_prefetch_disable                              0
        zfs_read_chunk_size                               1048576
        zfs_read_history                                  0
        zfs_read_history_hits                             0
        zfs_recover                                       0
        zfs_resilver_delay                                2
        zfs_resilver_min_time_ms                          3000
        zfs_scan_idle                                     50
        zfs_scan_min_time_ms                              1000
        zfs_scrub_delay                                   4
        zfs_send_corrupt_data                             0
        zfs_sync_pass_deferred_free                       2
        zfs_sync_pass_dont_compress                       5
        zfs_sync_pass_rewrite                             2
        zfs_sync_taskq_batch_pct                          75
        zfs_top_maxinflight                               32
        zfs_txg_history                                   0
        zfs_txg_timeout                                   5
        zfs_vdev_aggregation_limit                        131072
        zfs_vdev_async_read_max_active                    3
        zfs_vdev_async_read_min_active                    1
        zfs_vdev_async_write_active_max_dirty_percent     60
        zfs_vdev_async_write_active_min_dirty_percent     30
        zfs_vdev_async_write_max_active                   10
        zfs_vdev_async_write_min_active                   2
        zfs_vdev_cache_bshift                             16
        zfs_vdev_cache_max                                16384
        zfs_vdev_cache_size                               0
        zfs_vdev_max_active                               1000
        zfs_vdev_mirror_non_rotating_inc                  0
        zfs_vdev_mirror_non_rotating_seek_inc             1
        zfs_vdev_mirror_rotating_inc                      0
        zfs_vdev_mirror_rotating_seek_inc                 5
        zfs_vdev_mirror_rotating_seek_offset              1048576
        zfs_vdev_queue_depth_pct                          1000
        zfs_vdev_raidz_impl                               [fastest] original 
scalar sse2 ssse3 avx2
        zfs_vdev_read_gap_limit                           32768
        zfs_vdev_scheduler                                noop
        zfs_vdev_scrub_max_active                         2
        zfs_vdev_scrub_min_active                         1
        zfs_vdev_sync_read_max_active                     10
        zfs_vdev_sync_read_min_active                     10
        zfs_vdev_sync_write_max_active                    10
        zfs_vdev_sync_write_min_active                    10
        zfs_vdev_write_gap_limit                          4096
        zfs_zevent_cols                                   80
        zfs_zevent_console                                0
        zfs_zevent_len_max                                512
        zfs_zil_clean_taskq_maxalloc                      1048576
        zfs_zil_clean_taskq_minalloc                      1024
        zfs_zil_clean_taskq_nthr_pct                      100
        zil_replay_disable                                0
        zil_slog_bulk                                     786432
        zio_delay_max                                     30000
        zio_dva_throttle_enabled                          1
        zio_requeue_io_start_cut_in_line                  1
        zio_taskq_batch_pct                               75
        zvol_inhibit_dev                                  0
        zvol_major                                        230
        zvol_max_discard_blocks                           16384
        zvol_prefetch_bytes                               131072
        zvol_request_sync                                 0
        zvol_threads                                      32
        zvol_volmode                                      1

  
  Thanks

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: zfsutils-linux 0.7.5-1ubuntu16.4
  ProcVersionSignature: Ubuntu 4.15.0-39.42-generic 4.15.18
  Uname: Linux 4.15.0-39-generic x86_64
  NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
  ApportVersion: 2.20.9-0ubuntu7.5
  Architecture: amd64
  Date: Wed Feb  6 16:26:17 2019
  InstallationDate: Installed on 2016-04-04 (1038 days ago)
  InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Beta amd64 
(20160325)
  ProcEnviron:
   TERM=rxvt-unicode-256color
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: zfs-linux
  UpgradeStatus: Upgraded to bionic on 2018-08-16 (174 days ago)
  modified.conffile..etc.sudoers.d.zfs: [inaccessible: [Errno 13] Permission 
denied: '/etc/sudoers.d/zfs']

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1814983/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to