Public bug reported:
After switching to the 6.14 kernel (as shipped with linux-generic-
hwe-24.04 recently), ZFS read and write performance has seen a dramatic
drop.
Distributor ID: Ubuntu
Description: Ubuntu 24.04.2 LTS
Release: 24.04
Codename: noble
I've observed this on two separate ZFS baremetal hosts hosting different
disks and pool types (mirrored/striped and raidz2).
FIO Tests
-------
uname -r
6.14.0-27-generic
sudo fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1
--name=test --filename=$PWD/testfile --bs=1m --iodepth=64 --size=1G
--readwrite=read
test: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.36
Starting 1 process
Jobs: 1 (f=1): [R(1)][93.3%][r=76.9MiB/s][r=76 IOPS][eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=38786: Mon Aug 4 12:04:23 2025
read: IOPS=74, BW=74.7MiB/s (78.4MB/s)(1024MiB/13704msec)
bw ( KiB/s): min=18432, max=96256, per=100.00%, avg=77250.56,
stdev=17058.34, samples=25
iops : min= 18, max= 94, avg=75.44, stdev=16.66, samples=25
cpu : usr=0.18%, sys=7.70%, ctx=1028, majf=0, minf=16393
IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.1%, >=64=93.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=74.7MiB/s (78.4MB/s), 74.7MiB/s-74.7MiB/s (78.4MB/s-78.4MB/s),
io=1024MiB (1074MB), run=13704-13704msec
uname -r
6.8.0-71-generic
After switching back to 6.8 kernel
sudo fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1
--name=test --filename=$PWD/testfile --bs=1m --iodepth=64 --size=1G
--readwrite=read
test: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.36
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=5541: Mon Aug 4 12:14:13 2025
read: IOPS=3710, BW=3710MiB/s (3890MB/s)(1024MiB/276msec)
cpu : usr=0.00%, sys=99.64%, ctx=5, majf=0, minf=16393
IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.1%, >=64=93.8%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=3710MiB/s (3890MB/s), 3710MiB/s-3710MiB/s (3890MB/s-3890MB/s),
io=1024MiB (1074MB), run=276-276msec
** Affects: zfs-linux (Ubuntu)
Importance: Undecided
Status: New
** Affects: zfs-linux (Ubuntu Noble)
Importance: Undecided
Status: New
** Affects: zfs-linux (Ubuntu Plucky)
Importance: Undecided
Status: New
** Affects: zfs-linux (Ubuntu Questing)
Importance: Undecided
Status: New
** Also affects: zfs-linux (Ubuntu Noble)
Importance: Undecided
Status: New
** Also affects: zfs-linux (Ubuntu Plucky)
Importance: Undecided
Status: New
** Also affects: zfs-linux (Ubuntu Questing)
Importance: Undecided
Status: New
** Description changed:
After switching to the 6.14 kernel (as shipped with linux-generic-
hwe-24.04 recently), ZFS read and write performance has seen a dramatic
drop.
+ Distributor ID: Ubuntu
+ Description: Ubuntu 24.04.2 LTS
+ Release: 24.04
+ Codename: noble
+
I've observed this on two separate ZFS baremetal hosts hosting different
disks and pool types (mirrored/striped and raidz2).
-
FIO Tests
-------
uname -r
6.14.0-27-generic
sudo fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1
--name=test --filename=$PWD/testfile --bs=1m --iodepth=64 --size=1G
--readwrite=read
test: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.36
Starting 1 process
Jobs: 1 (f=1): [R(1)][93.3%][r=76.9MiB/s][r=76 IOPS][eta 00m:01s]
test: (groupid=0, jobs=1): err= 0: pid=38786: Mon Aug 4 12:04:23 2025
- read: IOPS=74, BW=74.7MiB/s (78.4MB/s)(1024MiB/13704msec)
- bw ( KiB/s): min=18432, max=96256, per=100.00%, avg=77250.56,
stdev=17058.34, samples=25
- iops : min= 18, max= 94, avg=75.44, stdev=16.66, samples=25
- cpu : usr=0.18%, sys=7.70%, ctx=1028, majf=0, minf=16393
- IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.1%, >=64=93.8%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
- complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
- issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=64
+ read: IOPS=74, BW=74.7MiB/s (78.4MB/s)(1024MiB/13704msec)
+ bw ( KiB/s): min=18432, max=96256, per=100.00%, avg=77250.56,
stdev=17058.34, samples=25
+ iops : min= 18, max= 94, avg=75.44, stdev=16.66, samples=25
+ cpu : usr=0.18%, sys=7.70%, ctx=1028, majf=0, minf=16393
+ IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.1%, >=64=93.8%
+ submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
+ complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
+ issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
+ latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
- READ: bw=74.7MiB/s (78.4MB/s), 74.7MiB/s-74.7MiB/s (78.4MB/s-78.4MB/s),
io=1024MiB (1074MB), run=13704-13704msec
-
+ READ: bw=74.7MiB/s (78.4MB/s), 74.7MiB/s-74.7MiB/s (78.4MB/s-78.4MB/s),
io=1024MiB (1074MB), run=13704-13704msec
uname -r
6.8.0-71-generic
After switching back to 6.8 kernel
sudo fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1
--name=test --filename=$PWD/testfile --bs=1m --iodepth=64 --size=1G
--readwrite=read
test: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T)
1024KiB-1024KiB, ioengine=libaio, iodepth=64
fio-3.36
Starting 1 process
test: (groupid=0, jobs=1): err= 0: pid=5541: Mon Aug 4 12:14:13 2025
- read: IOPS=3710, BW=3710MiB/s (3890MB/s)(1024MiB/276msec)
- cpu : usr=0.00%, sys=99.64%, ctx=5, majf=0, minf=16393
- IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.1%, >=64=93.8%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
- complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
- issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=64
+ read: IOPS=3710, BW=3710MiB/s (3890MB/s)(1024MiB/276msec)
+ cpu : usr=0.00%, sys=99.64%, ctx=5, majf=0, minf=16393
+ IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=0.8%, 16=1.6%, 32=3.1%, >=64=93.8%
+ submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
>=64=0.0%
+ complete : 0=0.0%, 4=99.9%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
+ issued rwts: total=1024,0,0,0 short=0,0,0,0 dropped=0,0,0,0
+ latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
- READ: bw=3710MiB/s (3890MB/s), 3710MiB/s-3710MiB/s (3890MB/s-3890MB/s),
io=1024MiB (1074MB), run=276-276msec
+ READ: bw=3710MiB/s (3890MB/s), 3710MiB/s-3710MiB/s (3890MB/s-3890MB/s),
io=1024MiB (1074MB), run=276-276msec
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2119470
Title:
ZFS I/O Performance degredation on 6.14 kernel
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/2119470/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
