Performing verification for Focal
I started a fresh VM with 5.4.0-144-generic from -updates. I attached 2x virtio
disks of 3gb each, for scratch disks.
I ran btrfs/154 with the following results:
# ./check btrfs/154
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 focal-xfs 5.4.0-144-generic #161-Ubuntu SMP Fri
Feb 3 14:49:04 UTC 2023
MKFS_OPTIONS -- /dev/vdd
MOUNT_OPTIONS -- /dev/vdd /scratch
btrfs/154 4s ... _check_dmesg: something found in dmesg (see
/home/ubuntu/xfstests-dev/results//btrfs/154.dmesg)
- output mismatch (see /home/ubuntu/xfstests-dev/results//btrfs/154.out.bad)
--- tests/btrfs/154.out 2023-01-28 07:54:34.007433164 +0000
+++ /home/ubuntu/xfstests-dev/results//btrfs/154.out.bad 2023-03-14
04:34:53.765899711 +0000
@@ -1,2 +1,6 @@
QA output created by 154
+Traceback (most recent call last):
+ File "/home/ubuntu/xfstests-dev/src/btrfs_crc32c_forged_name.py", line
99, in <module>
+ os.rename(srcpath, dstpath)
+OSError: [Errno 75] Value too large for defined data type: '/scratch/309'
->
b'/scratch/69f3?u\x97\xf3c33c58c648a2a9686fad82dbf43d7bfb443de4698f629e5b2b95126d6382430b8f29e4f502ccf306254d24cfd3800cb04a305989253db49f699a83cc2bc5d86a4f9b235891c0f72ba344a34e41aa69f819f196f7dbf29'
Silence is golden
...
(Run 'diff -u /home/ubuntu/xfstests-dev/tests/btrfs/154.out
/home/ubuntu/xfstests-dev/results//btrfs/154.out.bad' to see the entire diff)
Ran: btrfs/154
Failures: btrfs/154
Failed 1 of 1 tests
[ 49.889518] BTRFS info (device vdc): flagging fs with big metadata feature
[ 49.889520] BTRFS info (device vdc): disk space caching is enabled
[ 49.889521] BTRFS info (device vdc): has skinny extents
[ 49.891250] BTRFS info (device vdc): checking UUID tree
[ 50.007425] BTRFS: device fsid 382d436d-5f41-48a3-b96d-42c07ede9a03 devid 1
transid 5 /dev/vdd
[ 50.012807] BTRFS info (device vdd): flagging fs with big metadata feature
[ 50.012809] BTRFS info (device vdd): disk space caching is enabled
[ 50.012810] BTRFS info (device vdd): has skinny extents
[ 50.014307] BTRFS info (device vdd): checking UUID tree
[ 50.171099] BTRFS info (device vdc): flagging fs with big metadata feature
[ 50.171102] BTRFS info (device vdc): disk space caching is enabled
[ 50.171103] BTRFS info (device vdc): has skinny extents
[ 50.204928] run fstests btrfs/154 at 2023-03-14 04:34:47
[ 50.378091] BTRFS: device fsid 68eee97a-92e0-47da-9d5e-c6c8312ee358 devid 1
transid 5 /dev/vdd
[ 50.393188] BTRFS info (device vdd): flagging fs with big metadata feature
[ 50.393191] BTRFS info (device vdd): disk space caching is enabled
[ 50.393193] BTRFS info (device vdd): has skinny extents
[ 50.401657] BTRFS info (device vdd): checking UUID tree
[ 56.084117] ------------[ cut here ]------------
[ 56.084121] BTRFS: Transaction aborted (error -75)
[ 56.084229] WARNING: CPU: 2 PID: 1741 at fs/btrfs/inode.c:10148
btrfs_rename+0x9c6/0xa40 [btrfs]
[ 56.084265] CPU: 2 PID: 1741 Comm: python3 Not tainted 5.4.0-144-generic
#161-Ubuntu
[ 56.084267] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
1.16.0-debian-1.16.0-4 04/01/2014
[ 56.084302] RIP: 0010:btrfs_rename+0x9c6/0xa40 [btrfs]
[ 56.084306] Code: 48 0f ba a8 38 ce 00 00 02 72 25 41 83 f8 fb 74 43 41 83
f8 e2 74 3d 44 89 c6 48 c7 c7 e0 68 49 c0 44 89 45 a0 e8 ac 2c 0a e6 <0f> 0b 44
8b 45 a0 44 89 c1 ba a4 27 00 00 4c 89 f7 44 89 45 a0 48
[ 56.084309] RSP: 0018:ffffbb1182f17cd8 EFLAGS: 00010286
[ 56.084311] RAX: 0000000000000000 RBX: ffff9ca8b7114210 RCX: 0000000000000006
[ 56.084313] RDX: 0000000000000007 RSI: 0000000000000086 RDI: ffff9ca8bbb1c8c0
[ 56.084314] RBP: ffffbb1182f17d70 R08: 000000000000032b R09: 0000000000000004
[ 56.084315] R10: 0000000000000000 R11: 0000000000000001 R12: ffff9ca8b7104b40
[ 56.084317] R13: ffff9ca8b726a220 R14: ffff9ca8b7b03548 R15: 0000000000000236
[ 56.084320] FS: 00007f3e9e8be740(0000) GS:ffff9ca8bbb00000(0000)
knlGS:0000000000000000
[ 56.084321] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 56.084323] CR2: 00007fb85fbc5110 CR3: 000000016c17e004 CR4: 0000000000760ee0
[ 56.084331] PKRU: 55555554
[ 56.084333] Call Trace:
[ 56.084364] btrfs_rename2+0x1d/0x30 [btrfs]
[ 56.084371] vfs_rename+0x3df/0x9b0
[ 56.084377] ? _cond_resched+0x19/0x30
[ 56.084383] ? security_path_rename+0x88/0xb0
[ 56.084387] do_renameat2+0x507/0x570
[ 56.084391] __x64_sys_rename+0x23/0x30
[ 56.084397] do_syscall_64+0x57/0x190
[ 56.084401] entry_SYSCALL_64_after_hwframe+0x5c/0xc1
[ 56.084403] RIP: 0033:0x7f3e9eaece8b
[ 56.084406] Code: e8 ca ce 0a 00 85 c0 0f 95 c0 0f b6 c0 f7 d8 5d c3 66 0f
1f 44 00 00 b8 ff ff ff ff 5d c3 90 f3 0f 1e fa b8 52 00 00 00 0f 05 <48> 3d 00
f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 d1 8f 18 00 f7 d8
[ 56.084408] RSP: 002b:00007ffddfe043f8 EFLAGS: 00000246 ORIG_RAX:
0000000000000052
[ 56.084410] RAX: ffffffffffffffda RBX: 00007ffddfe044c0 RCX: 00007f3e9eaece8b
[ 56.084412] RDX: 0000000000000000 RSI: 00007f3e9e5584b0 RDI: 00007f3e9e676770
[ 56.084413] RBP: 00007ffddfe04470 R08: 0000000000000000 R09: 00000000ffffffff
[ 56.084414] R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffff9c
[ 56.084416] R13: 00000000ffffff9c R14: 00000000018c85c0 R15: 000000009e7c6700
[ 56.084419] ---[ end trace 800525b1e4cacc35 ]---
[ 56.084424] BTRFS: error (device vdd) in btrfs_rename:10148: errno=-75
unknown
[ 56.086071] BTRFS info (device vdd): forced readonly
[ 56.339439] BTRFS info (device vdd): flagging fs with big metadata feature
[ 56.339442] BTRFS info (device vdd): disk space caching is enabled
[ 56.339443] BTRFS info (device vdd): has skinny extents
[ 56.347861] BTRFS warning (device vdd): block group 30408704 has wrong
amount of free space
[ 56.347862] BTRFS warning (device vdd): failed to load free space cache for
block group 30408704, rebuilding it now
The test fails on the kernel in -updates, as expected.
I then enabled -proposed, and installed 5.4.0-145-generic.
Re-running the test on a clean btrfs filesystem:
# ./check btrfs/154
FSTYP -- btrfs
PLATFORM -- Linux/x86_64 focal-xfs 5.4.0-145-generic #162-Ubuntu SMP Fri
Feb 24 13:43:15 UTC 2023
MKFS_OPTIONS -- /dev/vdd
MOUNT_OPTIONS -- /dev/vdd /scratch
btrfs/154 4s ... 4s
Ran: btrfs/154
Passed all 1 tests
The test passes and there is no output in dmesg. The kernel in -proposed
fixes the issue and I am happy to mark it as verified for Focal.
** Tags removed: verification-needed-focal
** Tags added: verification-done-focal
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/2004132
Title:
btrfs/154: rename fails with EOVERFLOW when calculating item size
during item key collision
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Bionic:
Fix Committed
Status in linux source package in Focal:
Fix Committed
Bug description:
BugLink: https://bugs.launchpad.net/bugs/2004132
[Impact]
xfstests btrfs/154 fails on both Bionic and Focal, leading to a kernel
oops and the btrfs volume being forced readonly.
In btrfs, item key collision is allowed for some item types, namely
dir item and inode references. When inserting items into the btree,
there are two objects, the btrfs_item and the item data. These objects
must fit within the btree nodesize.
When a hash collision occurs, and we call btrfs_search_slot() to place
the objects in the tree, when btrfs_search_slot() reaches the leaf
node, a check is performed to see if we need to split the leaf. The
check is incorrect, returning that we need to split the leaf, since it
thinks that both btrfs_item and the item data need to be inserted,
when in reality, the item can be merged with the existing one and no
new btrfs_item will be inserted.
split_leaf() will return EOVERFLOW from following code:
if (extend && data_size + btrfs_item_size_nr(l, slot) +
sizeof(struct btrfs_item) > BTRFS_LEAF_DATA_SIZE(fs_info))
return -EOVERFLOW;
In the rename case, btrfs_check_dir_item_collision() is called early
stages of treewalking, and correctly calculates the needed size,
taking into account that a hash collision has occurred.
data_size = sizeof(*di) + name_len;
if (data_size + btrfs_item_size_nr(leaf, slot) +
sizeof(struct btrfs_item) > BTRFS_LEAF_DATA_SIZE(root->fs_info))
The two sizes reported from btrfs_check_dir_item_collision() and
btrfs_search_slot() are different, and rename fails due to
split_leaf() returning -EOVERFLOW, leading to transaction abort and
forcing the volume readonly.
Kernel oops:
BTRFS: Transaction aborted (error -75)
WARNING: CPU: 0 PID: 2921 at
/build/linux-fTmV3T/linux-4.15.0/fs/btrfs/inode.c:10217
btrfs_rename+0xcf1/0xdf0 [btrfs]
CPU: 0 PID: 2921 Comm: python3 Not tainted 4.15.0-202-generic #213-Ubuntu
RIP: 0010:btrfs_rename+0xcf1/0xdf0 [btrfs]
RSP: 0018:ffff9e6f4183fd20 EFLAGS: 00010282
RAX: 0000000000000000 RBX: ffff91a493f27b98 RCX: 0000000000000006
RDX: 0000000000000007 RSI: 0000000000000096 RDI: ffff91a4bfc1b4d0
RBP: ffff9e6f4183fdc0 R08: 00000000000002b4 R09: 0000000000000004
R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000236
R13: ffff91a493f56518 R14: ffff91a4b6b57b40 R15: ffff91a493f27b98
FS: 00007f6041081740(0000) GS:ffff91a4bfc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6040fe84c8 CR3: 000000015c8ca005 CR4: 0000000000760ef0
PKRU: 55555554
Call Trace:
btrfs_rename2+0x1d/0x30 [btrfs]
vfs_rename+0x46e/0x960
SyS_rename+0x362/0x3c0
do_syscall_64+0x73/0x130
entry_SYSCALL_64_after_hwframe+0x41/0xa6
Code: 0f ba a8 d0 cd 00 00 02 72 2b 41 83 f8 fb 0f 84 d9 00 00 00 44 89 c6 48
c7 c7 68 43 4b c0 44 89 55 80 44 89 45 98 e8 8f 5c a6 d0 <0f> 0b 44 8b 45 98 44
8b 55 80 44 89 55 80 44 89 c1 44 89 45 98
---[ end trace 9c6b87a19f4436f3 ]---
BTRFS: error (device vdd) in btrfs_rename:10217: errno=-75 unknown
BTRFS info (device vdd): forced readonly
[Testcase]
Start a fresh Bionic or Focal VM.
Attach two scratch disks, I used standard virtio disks with 3gb of
storage each. These disks are /dev/vdc and /dev/vdd.
Compile xfstests:
$ sudo apt-get install acl attr automake bc dbench dump e2fsprogs fio gawk \
gcc git indent libacl1-dev libaio-dev libcap-dev libgdbm-dev libtool \
libtool-bin libuuid1 lvm2 make psmisc python3 quota sed \
uuid-dev uuid-runtime xfsprogs linux-headers-$(uname -r) sqlite3 make
$ sudo apt-get install f2fs-tools ocfs2-tools udftools xfsdump \
xfslibs-dev
$ git clone git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
$ cd xfstests-dev
$ make
$ sudo su
# mkdir /test
# mkdir /scratch
# mkfs.btrfs -f /dev/vdc
# cat << EOF >> ./local.config
export TEST_DEV=/dev/vdc
export TEST_DIR=/test
export SCRATCH_DEV=/dev/vdd
export SCRATCH_MNT=/scratch
EOF
# ./check btrfs/154
btrfs/154 _check_dmesg: something found in dmesg (see
/home/ubuntu/xfstests-dev/results//btrfs/154.dmesg)
- output mismatch (see /home/ubuntu/xfstests-dev/results//btrfs/154.out.bad)
--- tests/btrfs/154.out 2023-01-28 02:53:03.566450703 +0000
+++ /home/ubuntu/xfstests-dev/results//btrfs/154.out.bad 2023-01-28
06:51:34.113121042 +0000
@@ -1,2 +1,6 @@
QA output created by 154
+Traceback (most recent call last):
+ File "/home/ubuntu/xfstests-dev/src/btrfs_crc32c_forged_name.py", line
99, in <module>
+ os.rename(srcpath, dstpath)
+OSError: [Errno 75] Value too large for defined data type:
'/scratch/309' ->
b'/scratch/fa5d\x90O\x97015d7a47c48fdeb99b857c17e8038da6382fcb05e3a6b367589a8f54a8c3c1327584dfa630b4bd8c5bbb37b4decc2b82fecb4c940e0bd0989166c44e6af2855e9e42a02a57cffa2fc5283525ac53b2b0d2baaf874ff50b'
Ran: btrfs/154
Failures: btrfs/154
Failed 1 of 1 tests
If you examine dmesg, you will see the oops from the impact section.
If you install the test kernel from the below ppa:
https://launchpad.net/~mruffell/+archive/ubuntu/sf349467-btrfs-154
The issue no longer occurs:
# ./check btrfs/154
Ran: btrfs/154
Passed all 1 tests
[Fix]
This was fixed in 5.11-rc3 by the below commit:
commit 9a664971569daf68254928149f580b4f5856d274
Author: ethanwu <etha...@synology.com>
Date: Tue Dec 1 17:25:12 2020 +0800
Subject: btrfs: correctly calculate item size used when item key collision
happens
Link:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9a664971569daf68254928149f580b4f5856d274
This cherry-picks to Focal, and required a small backport to Bionic,
removing the hunk that contained comments explaining the parameters to
btrfs_search_slot().
[Where problems could occur]
Problems could occur when calculating the size required for btrfs_item
and item data when hash collisions occur. Such collisions are rare in
of itself, but possible if you have a large amount files or craft
filenames to force collisions with the crc32 hash algorithm.
If a regression were to occur, it could cause more transactions to be
aborted, and would likely result in the users volume being forced read
only. They might not lose any existing data, but data being written
might be lost.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2004132/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
