Public bug reported: A btrfs linear, single volume consisting of 5 drives failed to scrub:
[ 1607.392691] BTRFS info (device sdg): scrub: started on devid 3 [ 1607.411805] BTRFS info (device sdg): scrub: started on devid 4 [ 1607.447140] BTRFS info (device sdg): scrub: started on devid 2 [ 1607.473112] BTRFS info (device sdg): scrub: started on devid 5 [ 1607.473125] BTRFS info (device sdg): scrub: started on devid 1 [ 3200.841468] BTRFS warning (device sdg): checksum error at logical 49377465106432 on dev /dev/sdc, physical 204288065536: metadata leaf (level 0) in tree 7 [ 3200.841474] BTRFS warning (device sdg): checksum error at logical 49377465106432 on dev /dev/sdc, physical 204288065536: metadata leaf (level 0) in tree 7 [ 3200.841479] BTRFS error (device sdg): bdev /dev/sdc errs: wr 510, rd 510, flush 0, corrupt 2314, gen 0 [ 3201.697768] BTRFS error (device sdg): unable to fixup (regular) error at logical 49377465106432 on dev /dev/sdc [ 6346.420285] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [ 6349.274210] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 17825815631468019255 [ 6349.283534] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [ 6349.283733] BTRFS info (device sdg): scrub: not finished on devid 5 with status: -5 [15791.355196] BTRFS warning (device sdg): checksum/header error at logical 49377465106432 on dev /dev/sdg, physical 2159525789696: metadata leaf (level 0) in tree 7 [15791.355201] BTRFS warning (device sdg): checksum/header error at logical 49377465106432 on dev /dev/sdg, physical 2159525789696: metadata leaf (level 0) in tree 7 [15791.355207] BTRFS error (device sdg): bdev /dev/sdg errs: wr 0, rd 0, flush 0, corrupt 0, gen 6 [15791.367367] BTRFS error (device sdg): unable to fixup (regular) error at logical 49377465106432 on dev /dev/sdg [20535.495225] BTRFS info (device sdg): scrub: finished on devid 1 with status: 0 [20535.805174] BTRFS info (device sdg): scrub: finished on devid 2 with status: 0 [20539.869609] BTRFS info (device sdg): scrub: finished on devid 3 with status: 0 [20603.754534] BTRFS info (device sdg): scrub: finished on devid 4 with status: 0 [21834.191123] BTRFS info (device sdg): scrub: started on devid 5 [21834.334527] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [21835.759240] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 17825815631468019255 [21835.759783] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [21835.759910] BTRFS info (device sdg): scrub: not finished on devid 5 with status: -5 The following command was advised to overcome the error: btrfs check --init-extent-tree /dev/sdg Now, the scrub crashes as follows: [15791.355196] BTRFS warning (device sdg): checksum/header error at logical 49377465106432 on dev /dev/sdg, physical 2159525789696: metadata leaf (level 0) i n tree 7 [15791.355201] BTRFS warning (device sdg): checksum/header error at logical 49377465106432 on dev /dev/sdg, physical 2159525789696: metadata leaf (level 0) i n tree 7 [15791.355207] BTRFS error (device sdg): bdev /dev/sdg errs: wr 0, rd 0, flush 0, corrupt 0, gen 6 [15791.367367] BTRFS error (device sdg): unable to fixup (regular) error at logical 49377465106432 on dev /dev/sdg [20535.495225] BTRFS info (device sdg): scrub: finished on devid 1 with status: 0 [20535.805174] BTRFS info (device sdg): scrub: finished on devid 2 with status: 0 [20539.869609] BTRFS info (device sdg): scrub: finished on devid 3 with status: 0 [20603.754534] BTRFS info (device sdg): scrub: finished on devid 4 with status: 0 [21834.191123] BTRFS info (device sdg): scrub: started on devid 5 [21834.334527] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [21835.759240] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 17825815631468019255 [21835.759783] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [21835.759910] BTRFS info (device sdg): scrub: not finished on devid 5 with status: -5 [21917.573684] BTRFS info (device sdg): scrub: started on devid 5 [21917.592098] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [21918.932051] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 48134072074240 [21918.935969] BTRFS error (device sdg): bad tree block start, want 49377465106432 have 17825815631468019255 [21918.971873] BTRFS info (device sdg): scrub: not finished on devid 5 with status: -5 [24594.433861] EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null) [24633.390103] EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null) [28804.075490] BTRFS info (device sdg): disk space caching is enabled [28804.075494] BTRFS info (device sdg): has skinny extents [28805.039000] BTRFS info (device sdg): bdev /dev/sdg errs: wr 0, rd 0, flush 0, corrupt 0, gen 6 [28805.039009] BTRFS info (device sdg): bdev /dev/sdf errs: wr 2274065, rd 17401701, flush 0, corrupt 0, gen 1 [28805.039016] BTRFS info (device sdg): bdev /dev/sde errs: wr 0, rd 0, flush 0, corrupt 90, gen 0 [28805.039021] BTRFS info (device sdg): bdev /dev/sdc errs: wr 510, rd 510, flush 0, corrupt 2314, gen 0 [28836.729425] BTRFS info (device sdg): scrub: started on devid 1 [32648.846447] BTRFS info (device sdg): scrub: not finished on devid 1 with status: -125 [73660.524782] BTRFS info (device sdg): disk space caching is enabled [73660.524785] BTRFS info (device sdg): has skinny extents [73661.366142] BTRFS info (device sdg): bdev /dev/sdg errs: wr 0, rd 0, flush 0, corrupt 0, gen 6 [73661.366150] BTRFS info (device sdg): bdev /dev/sdf errs: wr 2274065, rd 17401701, flush 0, corrupt 0, gen 1 [73661.366157] BTRFS info (device sdg): bdev /dev/sde errs: wr 0, rd 0, flush 0, corrupt 90, gen 0 [73661.366162] BTRFS info (device sdg): bdev /dev/sdc errs: wr 510, rd 510, flush 0, corrupt 2314, gen 0 [73661.542731] BTRFS info (device sdg): checking UUID tree [73690.995430] BTRFS info (device sdg): scrub: started on devid 1 [73691.985957] BTRFS error (device sdg): tree block 49392288530432 owner 3 already locked by pid=5334, extent tree corruption detected [73691.985971] ------------[ cut here ]------------ [73691.985973] BTRFS: Transaction aborted (error -117) [73691.986110] WARNING: CPU: 1 PID: 5334 at fs/btrfs/volumes.c:2987 btrfs_remove_chunk+0x6bc/0x700 [btrfs] [73691.986112] Modules linked in: nls_iso8859_1 snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg intel_rapl_msr snd_hda_codec snd_hda_core snd_hwdep mei_hdcp snd_pcm intel_rapl_common x86_pkg_temp_thermal intel_powerclamp kvm_intel snd_seq_midi snd_seq_midi_event kvm snd_rawmidi crct10dif_pclmul ghash_clmulni_intel cryptd joydev rapl intel_cstate snd_seq input_leds efi_pstore i915 snd_seq_device snd_timer mxm_wmi at24 snd drm_kms_helper cec soundcore rc_core mei_me i2c_algo_bit fb_sys_fops syscopyarea mei sysfillrect sysimgblt mac_hid intel_smartconnect sch_fq_codel nct6775 hwmon_vid coretemp parport_pc ppdev lp parport drm ip_tables x_tables autofs4 btrfs blake2b_generic xor raid6_pq libcrc32c hid_logitech_hidpp hid_logitech_dj hid_generic usbhid hid crc32_pclmul mpt3sas ahci xhci_pci libahci lpc_ich xhci_pci_renesas raid_class r8169 i2c_i801 i2c_smbus realtek scsi_transport_sas wmi video [73691.986190] CPU: 1 PID: 5334 Comm: btrfs-cleaner Not tainted 5.8.0-59-generic #66~20.04.1-Ubuntu [73691.986192] Hardware name: MSI MS-7835/C847MS-E33 (MS-7835), BIOS V1.1 03/07/2013 [73691.986256] RIP: 0010:btrfs_remove_chunk+0x6bc/0x700 [btrfs] [73691.986262] Code: 8b 57 50 f0 48 0f ba aa 40 0a 00 00 02 8b 45 a0 72 1d 83 f8 fb 74 43 83 f8 e2 74 3e 89 c6 48 c7 c7 28 19 57 c0 e8 dc 41 03 ce <0f> 0b 8b 45 a0 89 c1 ba ab 0b 00 00 4c 89 ff 89 45 a0 48 c7 c6 b0 [73691.986265] RSP: 0018:ffffc1ed4091fd68 EFLAGS: 00010286 [73691.986269] RAX: 0000000000000000 RBX: 0000000040000000 RCX: 0000000000000027 [73691.986271] RDX: 0000000000000027 RSI: 0000000000000086 RDI: ffff9e2617318cd8 [73691.986274] RBP: ffffc1ed4091fe18 R08: ffff9e2617318cd0 R09: 0000000000000004 [73691.986276] R10: 0000000000000000 R11: 0000000000000001 R12: ffff9e25fa7c8800 [73691.986278] R13: ffff9e256904bc60 R14: ffff9e26099d8388 R15: ffff9e257895bb60 [73691.986282] FS: 0000000000000000(0000) GS:ffff9e2617300000(0000) knlGS:0000000000000000 [73691.986285] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [73691.986288] CR2: 0000560b1491bd90 CR3: 0000000135c0a006 CR4: 00000000000606e0 [73691.986291] Call Trace: [73691.986363] btrfs_delete_unused_bgs+0x6f5/0x790 [btrfs] [73691.986416] cleaner_kthread+0x124/0x130 [btrfs] [73691.986424] kthread+0x114/0x150 [73691.986473] ? refcount_add_not_zero.constprop.0+0x50/0x50 [btrfs] [73691.986477] ? kthread_park+0x90/0x90 [73691.986484] ret_from_fork+0x22/0x30 [73691.986490] ---[ end trace 6ea90edff35b62cd ]--- [73691.986496] BTRFS: error (device sdg) in btrfs_remove_chunk:2987: errno=-117 Filesystem corrupted [73691.986502] BTRFS info (device sdg): forced readonly [73691.986540] ------------[ cut here ]------------ [73691.986605] WARNING: CPU: 1 PID: 5334 at fs/btrfs/space-info.h:116 btrfs_block_rsv_release+0x2f4/0x300 [btrfs] [73691.986606] Modules linked in: nls_iso8859_1 snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel snd_intel_dspcfg intel_rapl_msr snd_hda_codec snd_hda_core snd_hwdep mei_hdcp snd_pcm intel_rapl_common x86_pkg_temp_thermal intel_powerclamp kvm_intel snd_seq_midi snd_seq_midi_event kvm snd_rawmidi crct10dif_pclmul ghash_clmulni_intel cryptd joydev rapl intel_cstate snd_seq input_leds efi_pstore i915 snd_seq_device snd_timer mxm_wmi at24 snd drm_kms_helper cec soundcore rc_core mei_me i2c_algo_bit fb_sys_fops syscopyarea mei sysfillrect sysimgblt mac_hid intel_smartconnect sch_fq_codel nct6775 hwmon_vid coretemp parport_pc ppdev lp parport drm ip_tables x_tables autofs4 btrfs blake2b_generic xor raid6_pq libcrc32c hid_logitech_hidpp hid_logitech_dj hid_generic usbhid hid crc32_pclmul mpt3sas ahci xhci_pci libahci lpc_ich xhci_pci_renesas raid_class r8169 i2c_i801 i2c_smbus realtek scsi_transport_sas wmi video [73691.986669] CPU: 1 PID: 5334 Comm: btrfs-cleaner Tainted: G W 5.8.0-59-generic #66~20.04.1-Ubuntu [73691.986671] Hardware name: MSI MS-7835/C847MS-E33 (MS-7835), BIOS V1.1 03/07/2013 [73691.986733] RIP: 0010:btrfs_block_rsv_release+0x2f4/0x300 [btrfs] [73691.986737] Code: b2 6a 2b ce 49 8b 06 48 85 c0 75 d8 4c 8b 6d c0 4c 8b 75 b8 4c 8b 65 b0 4c 8b 5d a8 e9 af fe ff ff 4c 39 f0 0f 83 b6 fe ff ff <0f> 0b 31 c0 e9 b0 fe ff ff 0f 1f 00 66 66 66 66 90 48 85 f6 74 23 [73691.986740] RSP: 0018:ffffc1ed4091fcf0 EFLAGS: 00010287 [73691.986743] RAX: 000000000005c000 RBX: ffff9e26099d8000 RCX: 0000000000000001 [73691.986746] RDX: 0000000000000001 RSI: ffff9e26099d8170 RDI: ffff9e25f51cfc00 [73691.986748] RBP: ffffc1ed4091fd48 R08: ffff9e2617318cd0 R09: 0000000000000004 [73691.986750] R10: 0000000000000000 R11: 0000000000060000 R12: ffff9e25f51cfc00 [73691.986752] R13: 0000000000000000 R14: 0000000000060000 R15: 0000000000000000 [73691.986756] FS: 0000000000000000(0000) GS:ffff9e2617300000(0000) knlGS:0000000000000000 [73691.986759] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [73691.986761] CR2: 0000560b1491bd90 CR3: 0000000135c0a006 CR4: 00000000000606e0 [73691.986763] Call Trace: [73691.986818] btrfs_trans_release_chunk_metadata+0x35/0x50 [btrfs] [73691.986876] btrfs_create_pending_block_groups+0x140/0x240 [btrfs] [73691.986928] __btrfs_end_transaction+0x3f/0x1b0 [btrfs] [73691.986978] btrfs_end_transaction+0x10/0x20 [btrfs] [73691.987035] btrfs_delete_unused_bgs+0x339/0x790 [btrfs] [73691.987085] cleaner_kthread+0x124/0x130 [btrfs] [73691.987090] kthread+0x114/0x150 [73691.987138] ? refcount_add_not_zero.constprop.0+0x50/0x50 [btrfs] [73691.987142] ? kthread_park+0x90/0x90 [73691.987148] ret_from_fork+0x22/0x30 [73691.987153] ---[ end trace 6ea90edff35b62ce ]--- [73691.993753] BTRFS warning (device sdg): failed setting block group ro: -30 [73691.993759] BTRFS info (device sdg): scrub: not finished on devid 1 with status: -30 # apt list|grep btrfs apt-btrfs-snapshot/focal,focal 3.5.3 all btrfs-compsize/focal 1.3-2 amd64 btrfs-heatmap/focal,focal,now 8-1 all [installed] btrfs-progs/focal,now 5.4.1-2 amd64 [installed] btrfs-progs/focal 5.4.1-2 i386 btrfsmaintenance/focal,focal 0.4.2-1 all golang-github-containerd-btrfs-dev/focal,focal 0.0~git20200117.1539353-1 all libblockdev-btrfs-dev/focal 2.23-2ubuntu3 amd64 libblockdev-btrfs-dev/focal 2.23-2ubuntu3 i386 libblockdev-btrfs2/focal 2.23-2ubuntu3 amd64 libblockdev-btrfs2/focal 2.23-2ubuntu3 i386 libbtrfs-dev/focal 5.4.1-2 amd64 libbtrfs-dev/focal 5.4.1-2 i386 libbtrfs0/focal 5.4.1-2 amd64 libbtrfs0/focal 5.4.1-2 i386 libbtrfsutil-dev/focal 5.4.1-2 amd64 libbtrfsutil-dev/focal 5.4.1-2 i386 libbtrfsutil1/focal 5.4.1-2 amd64 libbtrfsutil1/focal 5.4.1-2 i386 monitoring-plugins-btrfs/focal,focal 11-2 all munin-plugins-btrfs/focal,focal 11-2 all python3-btrfs/focal,focal,now 11-2 all [installed,automatic] python3-btrfsutil/focal 5.4.1-2 amd64 python3-btrfsutil/focal 5.4.1-2 i386 udisks2-btrfs/focal 2.8.4-1ubuntu1 amd64 udisks2-btrfs/focal 2.8.4-1ubuntu1 i386 ** Affects: ubuntu Importance: Undecided Status: New ** Tags: btrfs scrub -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1937380 Title: btrfs fails during scrub To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+bug/1937380/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs