OK, Seems like we can resolve this by using some tuning. I was able to reproduce this on a single drive SSD pool configuration with 30 clones of the linux source and grep'ing for various strings.
So, the ARC determines that it can't free up enough memory by releasing unpinned buffers so the prune kicks in. With the use case you have here a large fraction of the ARC is pinned which causes the issue you see here. Tuning can resolve this for your specific use case. I recommend bumping the settings in: /sys/module/zfs/parameters/zfs_arc_meta_limit_percent /sys/module/zfs/parameters/zfs_arc_dnode_limit_percent For example, when I set these to 100% the performance improved dramatically and the pruning reduced. I recommend increasing these settings until you see performance running at the level you require. Try to avoid setting them too high - make sure they are set high enough for your use case but no higher. Let me know if this resolves your issue. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1814983 Title: zfs poor sustained read performance from ssd pool Status in Native ZFS for Linux: Unknown Status in zfs-linux package in Ubuntu: Incomplete Bug description: Hello, I'm seeing substantially slower read performance from an ssd pool than I expected. I have two pools on this computer; one ('fst') is four sata ssds, the other ('srv') is nine spinning metal drives. With a long-running ripgrep process on the fst pool, performance started out really good and grew to astonishingly good (iirc ~30kiops, as measured by zpool iostat -v 1). However after a few hours the performance has dropped to 30-40 iops. top reports an arc_reclaim and many arc_prune processes to be consuming most of the CPU time. I've included a screenshot of top, some output from zpool iostat -v 1, and arc_summary, with "===" to indicate the start of the next command's output: === top (memory in gigabytes): top - 16:27:53 up 70 days, 16:03, 3 users, load average: 35.67, 35.81, 35.58 Tasks: 809 total, 19 running, 612 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.0 us, 58.1 sy, 0.0 ni, 39.2 id, 2.6 wa, 0.0 hi, 0.0 si, 0.0 st GiB Mem : 125.805 total, 0.620 free, 96.942 used, 28.243 buff/cache GiB Swap: 5.694 total, 5.688 free, 0.006 used. 27.840 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1523 root 20 0 0.0m 0.0m 0.0m R 100.0 0.0 290:52.26 arc_reclaim 4484 root 20 0 0.0m 0.0m 0.0m R 56.2 0.0 1:18.79 arc_prune 6225 root 20 0 0.0m 0.0m 0.0m R 56.2 0.0 1:11.92 arc_prune 7601 root 20 0 0.0m 0.0m 0.0m S 56.2 0.0 2:50.25 arc_prune 30891 root 20 0 0.0m 0.0m 0.0m S 56.2 0.0 1:33.08 arc_prune 3057 root 20 0 0.0m 0.0m 0.0m S 55.9 0.0 9:00.95 arc_prune 3259 root 20 0 0.0m 0.0m 0.0m R 55.9 0.0 3:16.84 arc_prune 24008 root 20 0 0.0m 0.0m 0.0m S 55.9 0.0 1:55.71 arc_prune 1285 root 20 0 0.0m 0.0m 0.0m R 55.6 0.0 3:20.52 arc_prune 5345 root 20 0 0.0m 0.0m 0.0m R 55.6 0.0 1:15.99 arc_prune 30121 root 20 0 0.0m 0.0m 0.0m S 55.6 0.0 1:35.50 arc_prune 31192 root 20 0 0.0m 0.0m 0.0m S 55.6 0.0 6:17.16 arc_prune 32287 root 20 0 0.0m 0.0m 0.0m S 55.6 0.0 1:28.02 arc_prune 32625 root 20 0 0.0m 0.0m 0.0m R 55.6 0.0 1:27.34 arc_prune 22572 root 20 0 0.0m 0.0m 0.0m S 55.3 0.0 10:02.92 arc_prune 31989 root 20 0 0.0m 0.0m 0.0m R 55.3 0.0 1:28.03 arc_prune 3353 root 20 0 0.0m 0.0m 0.0m R 54.9 0.0 8:58.81 arc_prune 10252 root 20 0 0.0m 0.0m 0.0m R 54.9 0.0 2:36.37 arc_prune 1522 root 20 0 0.0m 0.0m 0.0m S 53.9 0.0 158:42.45 arc_prune 3694 root 20 0 0.0m 0.0m 0.0m R 53.9 0.0 1:20.79 arc_prune 13394 root 20 0 0.0m 0.0m 0.0m R 53.9 0.0 10:35.78 arc_prune 24592 root 20 0 0.0m 0.0m 0.0m R 53.9 0.0 1:54.19 arc_prune 25859 root 20 0 0.0m 0.0m 0.0m S 53.9 0.0 1:51.71 arc_prune 8194 root 20 0 0.0m 0.0m 0.0m S 53.6 0.0 0:54.51 arc_prune 18472 root 20 0 0.0m 0.0m 0.0m R 53.6 0.0 2:08.73 arc_prune 29525 root 20 0 0.0m 0.0m 0.0m R 53.6 0.0 1:35.81 arc_prune 32291 root 20 0 0.0m 0.0m 0.0m S 53.6 0.0 1:28.00 arc_prune 3156 root 20 0 0.0m 0.0m 0.0m R 53.3 0.0 3:17.68 arc_prune 6224 root 20 0 0.0m 0.0m 0.0m S 53.3 0.0 1:11.80 arc_prune 9788 root 20 0 0.0m 0.0m 0.0m S 53.3 0.0 0:46.00 arc_prune 10341 root 20 0 0.0m 0.0m 0.0m R 53.3 0.0 2:36.23 arc_prune 11881 root 20 0 0.0m 0.0m 0.0m S 53.0 0.0 2:31.57 arc_prune 24030 root 20 0 0.0m 0.0m 0.0m R 52.6 0.0 1:55.44 arc_prune === zpool iostat -v 1 output (for a while): capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 68 0 650K 0 mirror 588G 340G 31 0 331K 0 sdj - - 20 0 179K 0 sdk - - 10 0 152K 0 mirror 588G 340G 36 0 319K 0 sdl - - 17 0 132K 0 sdm - - 18 0 187K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 35 187K 144K mirror 443G 2.29T 0 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 63.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 0 17 0 71.8K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 5 0 23.9K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 5 0 23.9K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 5 0 23.9K mirror 443G 2.29T 1 17 124K 71.8K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 5 124K 23.9K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 5 0 23.9K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 5 0 23.9K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 110 0 1.07M 0 mirror 588G 340G 59 0 634K 0 sdj - - 28 0 303K 0 sdk - - 30 0 331K 0 mirror 588G 340G 50 0 459K 0 sdl - - 28 0 303K 0 sdm - - 21 0 155K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 229 183K 1.00M mirror 443G 2.29T 2 73 183K 335K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 2 24 183K 112K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 24 0 112K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 23 0 112K mirror 443G 2.29T 0 77 0 347K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 25 0 116K mirror 443G 2.29T 0 77 0 347K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 25 0 116K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 25 0 116K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 29 0 403K 0 mirror 588G 340G 12 0 171K 0 sdj - - 7 0 79.7K 0 sdk - - 4 0 91.7K 0 mirror 588G 340G 16 0 231K 0 sdl - - 6 0 128K 0 sdm - - 9 0 104K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 0 66 63.8K 359K mirror 443G 2.29T 0 21 0 120K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 6 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 7 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 6 0 39.9K mirror 443G 2.29T 0 21 0 120K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 7 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 6 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 6 0 39.9K mirror 443G 2.29T 0 22 63.8K 120K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 7 63.8K 39.9K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 6 0 39.9K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 7 0 39.9K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 97 0 797K 0 mirror 588G 340G 58 0 474K 0 sdj - - 27 0 263K 0 sdk - - 30 0 211K 0 mirror 588G 340G 38 0 323K 0 sdl - - 23 0 203K 0 sdm - - 14 0 120K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 176 187K 789K mirror 443G 2.29T 0 58 59.8K 263K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 19 59.8K 87.7K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 18 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 19 0 87.7K mirror 443G 2.29T 0 59 0 263K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 19 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 19 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 19 0 87.7K mirror 443G 2.29T 1 57 128K 263K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 18 128K 87.7K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 19 0 87.7K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 18 0 87.7K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 70 0 426K 0 mirror 588G 340G 38 0 263K 0 sdj - - 21 0 128K 0 sdk - - 16 0 135K 0 mirror 588G 340G 31 0 163K 0 sdl - - 10 0 67.7K 0 sdm - - 20 0 95.6K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 1 46 116K 2.36M mirror 443G 2.29T 0 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 59.8K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 0 37 0 2.31M ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 12 0 789K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 11 0 789K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 12 0 789K mirror 443G 2.29T 0 8 55.8K 47.8K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 0 2 55.8K 15.9K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 2 0 15.9K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 2 0 15.9K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 108 0 614K 0 mirror 588G 340G 50 0 299K 0 sdj - - 32 0 203K 0 sdk - - 17 0 95.6K 0 mirror 588G 340G 57 0 315K 0 sdl - - 30 0 155K 0 sdm - - 26 0 159K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 68 191K 311K mirror 443G 2.29T 0 8 0 47.8K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 2 0 15.9K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 2 0 15.9K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 2 0 15.9K mirror 443G 2.29T 0 29 0 132K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 9 0 43.8K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 9 0 43.8K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 9 0 43.8K mirror 443G 2.29T 2 29 191K 132K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 2 9 191K 43.8K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 9 0 43.8K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 9 0 43.8K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 66 0 379K 0 mirror 588G 340G 26 0 144K 0 sdj - - 12 0 63.8K 0 sdk - - 13 0 79.7K 0 mirror 588G 340G 39 0 235K 0 sdl - - 19 0 120K 0 sdm - - 19 0 116K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 166 183K 754K mirror 443G 2.29T 0 55 0 251K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 18 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 17 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 18 0 83.7K mirror 443G 2.29T 0 54 0 251K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 17 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 18 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 17 0 83.7K mirror 443G 2.29T 2 55 183K 251K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 2 18 183K 83.7K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 17 0 83.7K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 18 0 83.7K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 126 0 698K 0 mirror 588G 340G 64 0 335K 0 sdj - - 37 0 195K 0 sdk - - 26 0 140K 0 mirror 588G 340G 61 0 363K 0 sdl - - 34 0 207K 0 sdm - - 26 0 155K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 3 274 239K 1.23M mirror 443G 2.29T 1 91 120K 418K ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 1 30 120K 139K ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 29 0 139K ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 30 0 139K mirror 443G 2.29T 0 91 0 418K ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 30 0 139K ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 29 0 139K ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 30 0 139K mirror 443G 2.29T 1 91 120K 418K ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 30 120K 139K ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 30 0 139K ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 29 0 139K logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- capacity operations bandwidth pool alloc free read write read write ------------------------------------------- ----- ----- ----- ----- ----- ----- fst 1.15T 679G 70 0 442K 0 mirror 588G 340G 36 0 215K 0 sdj - - 18 0 95.6K 0 sdk - - 17 0 119K 0 mirror 588G 340G 33 0 227K 0 sdl - - 15 0 123K 0 sdm - - 17 0 104K 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- srv 1.30T 6.86T 2 0 187K 0 mirror 443G 2.29T 0 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P8KTWJYY - - 0 0 63.7K 0 ata-HGST_HUS724030ALA640_PN2234P9G620TW - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G66E2U - - 0 0 0 0 mirror 443G 2.29T 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69TKU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69TXU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G69U2U - - 0 0 0 0 mirror 443G 2.29T 1 0 123K 0 ata-HGST_HUS724030ALA640_PN2234P9G6EBUU - - 1 0 123K 0 ata-HGST_HUS724030ALA640_PN2234P9G6ESAU - - 0 0 0 0 ata-HGST_HUS724030ALA640_PN2234P9G6G70U - - 0 0 0 0 logs - - - - - - nvme0n1p1 900K 19.9G 0 0 0 0 cache - - - - - - nvme0n1p2 334G 764G 0 0 0 0 ------------------------------------------- ----- ----- ----- ----- ----- ----- === arc_summary ------------------------------------------------------------------------ ZFS Subsystem Report Wed Feb 06 16:34:09 2019 ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 142.80m Mutex Misses: 14.89m Evict Skips: 21.87b ARC Size: 86.88% 54.65 GiB Target Size: (Adaptive) 78.61% 49.45 GiB Min Size (Hard Limit): 6.25% 3.93 GiB Max Size (High Water): 16:1 62.90 GiB ARC Size Breakdown: Recently Used Cache Size: 25.05% 5.55 GiB Frequently Used Cache Size: 74.95% 16.62 GiB ARC Hash Breakdown: Elements Max: 7.46m Elements Current: 60.33% 4.50m Collisions: 43.79m Chain Max: 8 Chains: 504.81k ARC Total accesses: 2.51b Cache Hit Ratio: 95.24% 2.39b Cache Miss Ratio: 4.76% 119.32m Actual Hit Ratio: 92.99% 2.33b Data Demand Efficiency: 94.16% 486.53m Data Prefetch Efficiency: 86.47% 29.68m CACHE HITS BY CACHE LIST: Anonymously Used: 2.20% 52.51m Most Recently Used: 37.35% 891.38m Most Frequently Used: 60.29% 1.44b Most Recently Used Ghost: 0.10% 2.32m Most Frequently Used Ghost: 0.06% 1.45m CACHE HITS BY DATA TYPE: Demand Data: 19.20% 458.13m Prefetch Data: 1.08% 25.66m Demand Metadata: 78.22% 1.87b Prefetch Metadata: 1.51% 36.00m CACHE MISSES BY DATA TYPE: Demand Data: 23.80% 28.40m Prefetch Data: 3.37% 4.02m Demand Metadata: 66.03% 78.79m Prefetch Metadata: 6.80% 8.12m L2 ARC Summary: (HEALTHY) Low Memory Aborts: 233 Free on Write: 27.52k R/W Clashes: 0 Bad Checksums: 0 IO Errors: 0 L2 ARC Size: (Adaptive) 364.94 GiB Compressed: 91.59% 334.23 GiB Header Size: 0.08% 307.98 MiB L2 ARC Breakdown: 119.32m Hit Ratio: 1.42% 1.69m Miss Ratio: 98.58% 117.63m Feeds: 6.01m L2 ARC Writes: Writes Sent: 100.00% 279.55k DMU Prefetch Efficiency: 1.89b Hit Ratio: 2.24% 42.49m Miss Ratio: 97.76% 1.85b ZFS Tunable: dbuf_cache_hiwater_pct 10 dbuf_cache_lowater_pct 10 dbuf_cache_max_bytes 104857600 dbuf_cache_max_shift 5 dmu_object_alloc_chunk_shift 7 ignore_hole_birth 1 l2arc_feed_again 1 l2arc_feed_min_ms 200 l2arc_feed_secs 1 l2arc_headroom 2 l2arc_headroom_boost 200 l2arc_noprefetch 1 l2arc_norw 0 l2arc_write_boost 8388608 l2arc_write_max 8388608 metaslab_aliquot 524288 metaslab_bias_enabled 1 metaslab_debug_load 0 metaslab_debug_unload 0 metaslab_fragmentation_factor_enabled 1 metaslab_lba_weighting_enabled 1 metaslab_preload_enabled 1 metaslabs_per_vdev 200 send_holes_without_birth_time 1 spa_asize_inflation 24 spa_config_path /etc/zfs/zpool.cache spa_load_verify_data 1 spa_load_verify_maxinflight 10000 spa_load_verify_metadata 1 spa_slop_shift 5 zfetch_array_rd_sz 1048576 zfetch_max_distance 8388608 zfetch_max_streams 8 zfetch_min_sec_reap 2 zfs_abd_scatter_enabled 1 zfs_abd_scatter_max_order 10 zfs_admin_snapshot 1 zfs_arc_average_blocksize 8192 zfs_arc_dnode_limit 0 zfs_arc_dnode_limit_percent 10 zfs_arc_dnode_reduce_percent 10 zfs_arc_grow_retry 0 zfs_arc_lotsfree_percent 10 zfs_arc_max 0 zfs_arc_meta_adjust_restarts 4096 zfs_arc_meta_limit 0 zfs_arc_meta_limit_percent 75 zfs_arc_meta_min 0 zfs_arc_meta_prune 10000 zfs_arc_meta_strategy 1 zfs_arc_min 0 zfs_arc_min_prefetch_lifespan 0 zfs_arc_p_aggressive_disable 1 zfs_arc_p_dampener_disable 1 zfs_arc_p_min_shift 0 zfs_arc_pc_percent 0 zfs_arc_shrink_shift 0 zfs_arc_sys_free 0 zfs_autoimport_disable 1 zfs_compressed_arc_enabled 1 zfs_dbgmsg_enable 0 zfs_dbgmsg_maxsize 4194304 zfs_dbuf_state_index 0 zfs_deadman_checktime_ms 5000 zfs_deadman_enabled 1 zfs_deadman_synctime_ms 1000000 zfs_dedup_prefetch 0 zfs_delay_min_dirty_percent 60 zfs_delay_scale 500000 zfs_delete_blocks 20480 zfs_dirty_data_max 4294967296 zfs_dirty_data_max_max 4294967296 zfs_dirty_data_max_max_percent 25 zfs_dirty_data_max_percent 10 zfs_dirty_data_sync 67108864 zfs_dmu_offset_next_sync 0 zfs_expire_snapshot 300 zfs_flags 0 zfs_free_bpobj_enabled 1 zfs_free_leak_on_eio 0 zfs_free_max_blocks 100000 zfs_free_min_time_ms 1000 zfs_immediate_write_sz 32768 zfs_max_recordsize 1048576 zfs_mdcomp_disable 0 zfs_metaslab_fragmentation_threshold 70 zfs_metaslab_segment_weight_enabled 1 zfs_metaslab_switch_threshold 2 zfs_mg_fragmentation_threshold 85 zfs_mg_noalloc_threshold 0 zfs_multihost_fail_intervals 5 zfs_multihost_history 0 zfs_multihost_import_intervals 10 zfs_multihost_interval 1000 zfs_multilist_num_sublists 0 zfs_no_scrub_io 0 zfs_no_scrub_prefetch 0 zfs_nocacheflush 0 zfs_nopwrite_enabled 1 zfs_object_mutex_size 64 zfs_pd_bytes_max 52428800 zfs_per_txg_dirty_frees_percent 30 zfs_prefetch_disable 0 zfs_read_chunk_size 1048576 zfs_read_history 0 zfs_read_history_hits 0 zfs_recover 0 zfs_resilver_delay 2 zfs_resilver_min_time_ms 3000 zfs_scan_idle 50 zfs_scan_min_time_ms 1000 zfs_scrub_delay 4 zfs_send_corrupt_data 0 zfs_sync_pass_deferred_free 2 zfs_sync_pass_dont_compress 5 zfs_sync_pass_rewrite 2 zfs_sync_taskq_batch_pct 75 zfs_top_maxinflight 32 zfs_txg_history 0 zfs_txg_timeout 5 zfs_vdev_aggregation_limit 131072 zfs_vdev_async_read_max_active 3 zfs_vdev_async_read_min_active 1 zfs_vdev_async_write_active_max_dirty_percent 60 zfs_vdev_async_write_active_min_dirty_percent 30 zfs_vdev_async_write_max_active 10 zfs_vdev_async_write_min_active 2 zfs_vdev_cache_bshift 16 zfs_vdev_cache_max 16384 zfs_vdev_cache_size 0 zfs_vdev_max_active 1000 zfs_vdev_mirror_non_rotating_inc 0 zfs_vdev_mirror_non_rotating_seek_inc 1 zfs_vdev_mirror_rotating_inc 0 zfs_vdev_mirror_rotating_seek_inc 5 zfs_vdev_mirror_rotating_seek_offset 1048576 zfs_vdev_queue_depth_pct 1000 zfs_vdev_raidz_impl [fastest] original scalar sse2 ssse3 avx2 zfs_vdev_read_gap_limit 32768 zfs_vdev_scheduler noop zfs_vdev_scrub_max_active 2 zfs_vdev_scrub_min_active 1 zfs_vdev_sync_read_max_active 10 zfs_vdev_sync_read_min_active 10 zfs_vdev_sync_write_max_active 10 zfs_vdev_sync_write_min_active 10 zfs_vdev_write_gap_limit 4096 zfs_zevent_cols 80 zfs_zevent_console 0 zfs_zevent_len_max 512 zfs_zil_clean_taskq_maxalloc 1048576 zfs_zil_clean_taskq_minalloc 1024 zfs_zil_clean_taskq_nthr_pct 100 zil_replay_disable 0 zil_slog_bulk 786432 zio_delay_max 30000 zio_dva_throttle_enabled 1 zio_requeue_io_start_cut_in_line 1 zio_taskq_batch_pct 75 zvol_inhibit_dev 0 zvol_major 230 zvol_max_discard_blocks 16384 zvol_prefetch_bytes 131072 zvol_request_sync 0 zvol_threads 32 zvol_volmode 1 Thanks ProblemType: Bug DistroRelease: Ubuntu 18.04 Package: zfsutils-linux 0.7.5-1ubuntu16.4 ProcVersionSignature: Ubuntu 4.15.0-39.42-generic 4.15.18 Uname: Linux 4.15.0-39-generic x86_64 NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair ApportVersion: 2.20.9-0ubuntu7.5 Architecture: amd64 Date: Wed Feb 6 16:26:17 2019 InstallationDate: Installed on 2016-04-04 (1038 days ago) InstallationMedia: Ubuntu-Server 16.04 LTS "Xenial Xerus" - Beta amd64 (20160325) ProcEnviron: TERM=rxvt-unicode-256color PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash SourcePackage: zfs-linux UpgradeStatus: Upgraded to bionic on 2018-08-16 (174 days ago) modified.conffile..etc.sudoers.d.zfs: [inaccessible: [Errno 13] Permission denied: '/etc/sudoers.d/zfs'] To manage notifications about this bug go to: https://bugs.launchpad.net/zfs/+bug/1814983/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
