Verification done for focal-proposed.
No regressions in the consistent/flaky fail sets.

The flaky fail set had a long tail (tests that failed just a few times),
but it can be isolated to another btrfs commit, and it doesn't seem to
happen often. (e.g. not seen on focal-proposed, but seen on focal orig
kernel on master-next, and both have the commit.)
Thus not a general/permanent issue. More details in the next comment.

Proposed kernel:
---

$ uname -rv
4.15.0-125-generic #128-Ubuntu SMP Mon Nov 9 20:45:41 UTC 2020

$ VERSION='4.15.0-125-generic #128-Ubuntu'

Number of runs:

$ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep -c 
Failures:
14
$ RUNS=14

Consistent fail set:

$ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep 
^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w 
$RUNS
     14 btrfs/186
     14 btrfs/193
     14 btrfs/200
     14 btrfs/202
     14 btrfs/204
     14 btrfs/205
     14 btrfs/214
     14 btrfs/219
     14 btrfs/220
     14 btrfs/221
     14 btrfs/222
     14 generic/260
     14 generic/286
     14 generic/301
     14 generic/371
     14 generic/405
     14 generic/407
     14 generic/465
     14 generic/480
     14 generic/488
     14 generic/502
     14 generic/513
     14 generic/526
     14 generic/527
     14 generic/546
     14 generic/562
     14 generic/565

Flaky fail set:

$ grep -h -e ^Fail -e ^PLATFORM xfstests.log.* | grep -A1 "$VERSION" | grep 
^Failures: | cut -d' ' -f2- | tr ' ' '\n' | sort | uniq -c | sort | grep -w -v 
$RUNS
      1 btrfs/001  
      1 btrfs/147  
      1 btrfs/148  
      1 btrfs/149  
      1 btrfs/150  
      1 btrfs/151  
      1 btrfs/152  
      1 btrfs/153  
      1 btrfs/155  
      1 btrfs/156  
      1 btrfs/159  
      1 btrfs/164  
      1 btrfs/165  
      1 btrfs/166  
      1 btrfs/168  
      1 btrfs/169  
      1 btrfs/170  
      1 btrfs/171  
      1 btrfs/172  
      1 btrfs/173  
      1 btrfs/174  
      1 btrfs/177  
      1 btrfs/178  
      1 btrfs/179  
      1 btrfs/180  
      1 btrfs/181  
      1 btrfs/182  
      1 btrfs/183  
      1 btrfs/184  
      1 btrfs/187  
      1 btrfs/188  
      1 btrfs/190  
      1 btrfs/196  
      1 btrfs/197  
      1 btrfs/199  
      1 btrfs/206  
      1 btrfs/207  
      1 btrfs/208  
      1 btrfs/213  
      1 btrfs/217 
      1 generic/003
      1 generic/017
      1 generic/031
      1 generic/033
      1 generic/037
      1 generic/038
      1 generic/039
      1 generic/040
      1 generic/041
      1 generic/043
      1 generic/044
      1 generic/045
      1 generic/046
      1 generic/047
      1 generic/048
      1 generic/049
      1 generic/050
      1 generic/051
      1 generic/052
      1 generic/053
      1 generic/054
      1 generic/055
      1 generic/057
      1 generic/059
      1 generic/062
      1 generic/064
      1 generic/065
      1 generic/066
      1 generic/067
      1 generic/068
      1 generic/069
      1 generic/071
      1 generic/073
      1 generic/076
      1 generic/077
      1 generic/079
      1 generic/081
      1 generic/082
      1 generic/083
      1 generic/084
      1 generic/090
      1 generic/103
      1 generic/104
      1 generic/297
      1 generic/324
      1 generic/325
      1 generic/464
      1 generic/518
      2 btrfs/106  
      2 btrfs/107 
      2 btrfs/110 
      2 btrfs/111
      2 btrfs/112
      2 btrfs/114
      2 btrfs/115
      2 btrfs/116
      2 btrfs/117
      2 btrfs/118
      2 btrfs/119
      2 btrfs/120
      2 btrfs/121
      2 btrfs/122
      2 btrfs/123
      2 btrfs/126
      2 btrfs/127
      2 btrfs/128
      2 btrfs/129  
      2 btrfs/130  
      2 btrfs/131  
      2 btrfs/132  
      2 btrfs/133  
      2 btrfs/134  
      2 btrfs/135  
      2 btrfs/136  
      2 btrfs/137
      2 btrfs/138
      2 btrfs/139
      2 btrfs/144
      2 btrfs/145
      2 btrfs/146
      2 btrfs/189
      2 btrfs/191
      2 generic/056
      2 generic/085
      2 generic/089
      2 generic/101
      2 generic/102
      2 generic/509
      2 generic/510
      2 generic/601
      2 generic/602
      2 generic/603
      2 generic/604
      2 generic/605
      2 generic/606
      2 generic/607
      2 generic/608
      2 generic/610
      2 generic/611
      2 shared/002
      2 shared/032
      3 btrfs/076
      3 btrfs/077
      3 btrfs/079
      3 btrfs/080
      3 btrfs/082
      3 btrfs/083
      3 btrfs/084
      3 btrfs/085
      3 btrfs/086
      3 btrfs/087
      3 btrfs/088
      3 btrfs/089
      3 btrfs/090
      3 btrfs/091
      3 btrfs/092
      3 btrfs/093  
      3 btrfs/094  
      3 btrfs/095  
      3 btrfs/096  
      3 btrfs/097  
      3 btrfs/098  
      3 btrfs/099  
      3 btrfs/102  
      3 btrfs/103
      3 btrfs/104
      3 btrfs/105
      3 btrfs/108
      3 btrfs/109
      3 btrfs/113
      3 btrfs/210
      3 btrfs/211
      3 btrfs/212  
      3 btrfs/215  
      3 btrfs/224  
      3 btrfs/226  
      3 generic/015
      3 generic/018
      3 generic/027
      3 generic/029
      3 generic/030
      3 generic/032
      3 generic/034
      3 generic/600
      4 btrfs/057
      4 btrfs/058
      4 btrfs/059
      4 btrfs/060
      4 btrfs/061
      4 btrfs/062
      4 btrfs/063
      4 btrfs/064
      4 btrfs/065
      4 btrfs/066
      4 btrfs/067
      4 btrfs/068
      4 btrfs/069
      4 btrfs/070
      4 btrfs/071
      4 btrfs/072
      4 btrfs/073
      4 btrfs/074
      4 btrfs/075
      4 btrfs/081
      4 btrfs/203
      4 btrfs/209
      4 generic/520
      4 generic/589
      4 generic/590
      4 generic/592
      4 generic/593
      4 generic/594
      4 generic/595
      4 generic/599
      5 btrfs/042
      5 btrfs/043
      5 btrfs/044
      5 btrfs/045
      5 btrfs/046
      5 btrfs/047
      5 btrfs/048
      5 btrfs/049
      5 btrfs/050
      5 btrfs/051
      5 btrfs/052
      5 btrfs/053
      5 btrfs/054
      5 generic/476
      5 generic/584
      5 generic/585
      5 generic/588
      6 btrfs/035
      6 btrfs/036
      6 btrfs/037
      6 btrfs/038
      6 btrfs/039
      6 btrfs/040
      6 btrfs/041
      6 btrfs/055
      6 btrfs/056
      6 generic/322
      6 generic/573
      6 generic/574
      6 generic/575
      6 generic/576
      6 generic/577
      6 generic/579
      6 generic/580
      6 generic/581
      6 generic/582
      6 generic/583
      7 btrfs/026
      7 btrfs/027
      7 btrfs/029
      7 btrfs/030
      7 btrfs/031
      7 btrfs/032
      7 btrfs/033
      7 btrfs/034
      7 generic/531
      7 generic/570
      7 generic/572
      8 btrfs/028
      8 btrfs/201
      8 generic/501
      8 generic/563
      8 generic/566
      8 generic/567
      8 generic/569
     11 btrfs/078
     11 generic/175 << increase, seen w/ test/patched kernel; from other btrfs 
patches. (comment #7)
     13 btrfs/010
     13 generic/074
     13 generic/133
     13 generic/142
     13 generic/143
     13 generic/475

** Tags removed: verification-needed-bionic
** Tags added: verification-done-bionic

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1902254

Title:
  Bionic: btrfs: kernel BUG at /build/linux-
  eTBZpZ/linux-4.15.0/fs/btrfs/ctree.c:3233!

Status in linux package in Ubuntu:
  Fix Committed
Status in linux source package in Bionic:
  Fix Committed
Status in linux source package in Focal:
  Fix Committed
Status in linux source package in Groovy:
  Fix Committed

Bug description:
  [Impact]

   * Users of btrfs started hitting a kernel BUG() (below)
     after upgrade from 4.15.0-99.100 to 4.15.0-109.110,
     which has 55 btrfs changes.

       kernel BUG at /build/linux-eTBZpZ/linux-4.15.0/fs/btrfs/ctree.c:3233!
       ...
       Krnl PSW : 00000000be9cb874 00000000ef3786e8 
(btrfs_set_item_key_safe+0x152/0x1c0 [btrfs])
       ...
       [...] Call Trace:
       [...] btrfs_set_item_key_safe+0x11c/0x1c0 [btrfs])
       [...] __btrfs_drop_extents+0xb5a/0xda8 [btrfs]
       [...] btrfs_log_changed_extents+0x35c/0xaf0 [btrfs]
       [...] btrfs_log_inode+0x9ee/0x1080 [btrfs]
       [...] btrfs_log_inode_parent+0x224/0xa10 [btrfs]
       [...] btrfs_log_dentry_safe+0x80/0xa8 [btrfs]
       [...] btrfs_sync_file+0x392/0x550 [btrfs]
       [...] do_fsync+0x5e/0x90
       [...] SyS_fdatasync+0x32/0x48
       [...] system_call+0xd8/0x2c8

       $ git log --oneline Ubuntu-4.15.0-99.100..Ubuntu-4.15.0-109.110 -- 
fs/btrfs/ | wc -l
       55

   * The error happens at random moments, regardless of a
     particular activity/load. Workaround is to downgrade.

  [Fix]

   * This BUG()/function is addressed in patch 4/4 [1] of series
     'btrfs: Enhanced runtime defence against fuzzed images' [2],
     after issues in the real world, not just crafted fs images:
     'one internal report has hit one BUG_ON() with real world fs'
   
       kernel BUG at fs/btrfs/ctree.c:3188!
       ...
       RIP: 0010:btrfs_set_item_key_safe+0x16c/0x180
   
   * The patch/set [3] is applied in v5.10-rc1 and Ubuntu Unstable:
     - d16c702fe4f2 btrfs: ctree: check key order before merging tree blocks
     - 07cce5cf3b48 btrfs: extent-tree: kill the BUG_ON() in 
insert_inline_extent_backref()
     - 1c2a07f598d5 btrfs: extent-tree: kill BUG_ON() in __btrfs_free_extent()
     - f98b6215d7d1 btrfs: extent_io: do extra check for extent buffer read 
write functions
   
  [Test Case]

   * There is working synthetic reproducer for this issue,
     which is hard to reproduce as reported in commit [4]
     that introduces debugging for the issue.
     
   * Regression tests with xfstests and stress-ng shows
     no regressions between un/patched kernels.

  [Other Info]

   * Trivial backports (only refreshing a few context lines)
     with 3 more dependency patches on Bionic and 1 on Focal.
     And Bionic needed one extra hunk to '#include' a header.
     Groovy all apply cleanly.

  [1] https://lore.kernel.org/linux-btrfs/20200819063550.62832-5-...@suse.com/
  [2] https://lore.kernel.org/linux-btrfs/20200819063550.62832-1-...@suse.com/
  [3] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d16c702fe4f274bd77b47d3ab737eadcf24e0b93
  [4] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=7c15d41016dc886cc011e3854d855e219759ae68

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1902254/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to