Groovy/xfstests ====== Original/patched kernel versions: $ grep -h ^PLATFORM xfstests.log.* | sort -u PLATFORM -- Linux/s390x mfo-s390x-groovy 5.8.0-25-generic #26+test20201026b1 SMP Mon Oct 26 19:02:54 -03 2020 PLATFORM -- Linux/s390x mfo-s390x-groovy 5.8.0-25-generic #26-Ubuntu SMP Thu Oct 15 10:28:46 UTC 2020
Original kernel: --- $ VERSION='5.8.0-25-generic #26-Ubuntu' Number of runs: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep -c Failures: 12 $ RUNS=12 Consistent fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w $RUNS 12 btrfs/153 12 btrfs/208 12 btrfs/213 12 btrfs/219 12 btrfs/220 12 btrfs/221 12 btrfs/222 12 generic/260 12 generic/286 <<-- Fails 15/18 in patched kernel 12 generic/301 12 generic/465 <<-- Fails 15/18 in patched kernel 12 generic/610 Flaky fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w -v $RUNS 1 generic/297 1 generic/371 1 generic/528 2 generic/298 3 generic/166 10 btrfs/010 10 generic/074 10 generic/142 10 generic/143 11 generic/133 11 generic/175 11 generic/475 Patched kernel: --- $ VERSION='5.8.0-25-generic #26+test20201026b1' Number of runs: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep -c Failures: 18 $ RUNS=18 Consistent fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w $RUNS 18 btrfs/153 18 btrfs/208 18 btrfs/213 18 btrfs/219 18 btrfs/220 18 btrfs/221 18 btrfs/222 18 generic/260 18 generic/301 18 generic/610 No regressions. Flaky fail set: $ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep ^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w -v $RUNS 1 btrfs/020 1 btrfs/028 1 generic/118 1 generic/198 1 generic/215 1 generic/241 1 generic/246 1 generic/297 1 generic/371 1 generic/564 2 generic/298 3 generic/247 5 generic/166 11 btrfs/010 11 generic/074 11 generic/142 11 generic/143 13 generic/475 14 generic/175 15 generic/286 15 generic/465 16 generic/133 No regressions. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1902254 Title: Bionic: btrfs: kernel BUG at /build/linux- eTBZpZ/linux-4.15.0/fs/btrfs/ctree.c:3233! Status in linux package in Ubuntu: Fix Committed Status in linux source package in Bionic: In Progress Status in linux source package in Focal: In Progress Status in linux source package in Groovy: In Progress Bug description: [Impact] * Users of btrfs started hitting a kernel BUG() (below) after upgrade from 4.15.0-99.100 to 4.15.0-109.110, which has 55 btrfs changes. kernel BUG at /build/linux-eTBZpZ/linux-4.15.0/fs/btrfs/ctree.c:3233! ... Krnl PSW : 00000000be9cb874 00000000ef3786e8 (btrfs_set_item_key_safe+0x152/0x1c0 [btrfs]) ... [...] Call Trace: [...] btrfs_set_item_key_safe+0x11c/0x1c0 [btrfs]) [...] __btrfs_drop_extents+0xb5a/0xda8 [btrfs] [...] btrfs_log_changed_extents+0x35c/0xaf0 [btrfs] [...] btrfs_log_inode+0x9ee/0x1080 [btrfs] [...] btrfs_log_inode_parent+0x224/0xa10 [btrfs] [...] btrfs_log_dentry_safe+0x80/0xa8 [btrfs] [...] btrfs_sync_file+0x392/0x550 [btrfs] [...] do_fsync+0x5e/0x90 [...] SyS_fdatasync+0x32/0x48 [...] system_call+0xd8/0x2c8 $ git log --oneline Ubuntu-4.15.0-99.100..Ubuntu-4.15.0-109.110 -- fs/btrfs/ | wc -l 55 * The error happens at random moments, regardless of a particular activity/load. Workaround is to downgrade. [Fix] * This BUG()/function is addressed in patch 4/4 [1] of series 'btrfs: Enhanced runtime defence against fuzzed images' [2], after issues in the real world, not just crafted fs images: 'one internal report has hit one BUG_ON() with real world fs' kernel BUG at fs/btrfs/ctree.c:3188! ... RIP: 0010:btrfs_set_item_key_safe+0x16c/0x180 * The patch/set [3] is applied in v5.10-rc1 and Ubuntu Unstable: - d16c702fe4f2 btrfs: ctree: check key order before merging tree blocks - 07cce5cf3b48 btrfs: extent-tree: kill the BUG_ON() in insert_inline_extent_backref() - 1c2a07f598d5 btrfs: extent-tree: kill BUG_ON() in __btrfs_free_extent() - f98b6215d7d1 btrfs: extent_io: do extra check for extent buffer read write functions [Test Case] * There is working synthetic reproducer for this issue, which is hard to reproduce as reported in commit [4] that introduces debugging for the issue. * Regression tests with xfstests and stress-ng shows no regressions between un/patched kernels. [Other Info] * Trivial backports (only refreshing a few context lines) with 3 more dependency patches on Bionic and 1 on Focal. And Bionic needed one extra hunk to '#include' a header. Groovy all apply cleanly. [1] https://lore.kernel.org/linux-btrfs/20200819063550.62832-5-...@suse.com/ [2] https://lore.kernel.org/linux-btrfs/20200819063550.62832-1-...@suse.com/ [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d16c702fe4f274bd77b47d3ab737eadcf24e0b93 [4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7c15d41016dc886cc011e3854d855e219759ae68 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1902254/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp