ubuntu@ubuntu:~$ uname -r 4.15.0-56-generic ubuntu@ubuntu:~$ cat /proc/version Linux version 4.15.0-56-generic (arighi@kathleen) (gcc version 7.4.0 (Ubuntu 7.4.0-1ubuntu1~18.04.1)) #62~lp1796292 SMP Thu Aug 1 07:45:21 UTC 2019
This failed on the second install while running bcache-super-show /dev/vdg Hung task. [ 259.150347] bcache: run_cache_set() invalidating existing data [ 259.158038] bcache: register_cache() registered cache device nvme1n1p2 [ 259.251093] bcache: register_bdev() registered backing device vdg [ 259.379809] bcache: bch_cached_dev_attach() Caching vdg as bcache3 on set 084505ad-5f6c-4666-9e3e-4f1650e8b015 [ 259.411486] bcache: register_bcache() error /dev/vdg: device already registered (emitting change event) [ 259.537070] bcache: register_bcache() error /dev/vdg: device already registered (emitting change event) [ 259.797830] bcache: register_bcache() error /dev/vdg: device already registered (emitting change event) [ 259.900392] bcache: register_bcache() error /dev/vdg: device already registered (emitting change event) [ 271.682662] md: md0: resync done. [ 484.525529] INFO: task python3:11257 blocked for more than 120 seconds. [ 484.528933] Tainted: P O 4.15.0-56-generic #62~lp1796292 [ 484.532221] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 484.535936] python3 D 0 11257 7974 0x00000000 [ 484.535941] Call Trace: [ 484.535952] __schedule+0x291/0x8a0 [ 484.535957] schedule+0x2c/0x80 [ 484.535977] bch_bucket_alloc+0x1fa/0x350 [bcache] [ 484.535984] ? wait_woken+0x80/0x80 [ 484.535993] __bch_bucket_alloc_set+0xfe/0x150 [bcache] [ 484.536002] bch_bucket_alloc_set+0x4e/0x70 [bcache] [ 484.536014] __uuid_write+0x59/0x150 [bcache] [ 484.536025] ? __write_super+0x137/0x170 [bcache] [ 484.536035] bch_uuid_write+0x16/0x40 [bcache] [ 484.536046] __cached_dev_store+0x1d8/0x8a0 [bcache] [ 484.536057] bch_cached_dev_store+0x39/0xc0 [bcache] [ 484.536061] sysfs_kf_write+0x3c/0x50 [ 484.536064] kernfs_fop_write+0x125/0x1a0 [ 484.536069] __vfs_write+0x1b/0x40 [ 484.536071] vfs_write+0xb1/0x1a0 [ 484.536075] SyS_write+0x5c/0xe0 [ 484.536081] do_syscall_64+0x73/0x130 [ 484.536085] entry_SYSCALL_64_after_hwframe+0x3d/0xa2 [ 484.536088] RIP: 0033:0x7f2ac7aa7154 [ 484.536090] RSP: 002b:00007ffff157b628 EFLAGS: 00000246 ORIG_RAX: 0000000000000001 [ 484.536093] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f2ac7aa7154 [ 484.536095] RDX: 0000000000000008 RSI: 0000000001c96340 RDI: 0000000000000003 [ 484.536096] RBP: 00007f2ac7fa26c0 R08: 0000000000000000 R09: 0000000000000000 [ 484.536098] R10: 0000000000000100 R11: 0000000000000246 R12: 0000000000000003 [ 484.536099] R13: 0000000000000000 R14: 0000000001c96340 R15: 00000000019c7db0 I'll note that, I can run bcache-super-show on the device while this was hung. $ sudo bash root@ubuntu:~# bcache-super-show /dev/vdg sb.magic ok sb.first_sector 8 [match] sb.csum C71A896B52F1C486 [match] sb.version 1 [backing device] dev.label osddata5 dev.uuid 11df8370-fb64-4bb0-8171-dadabb47f6b1 dev.sectors_per_block 1 dev.sectors_per_bucket 1024 dev.data.first_sector 16 dev.data.cache_mode 1 [writeback] dev.data.cache_state 1 [clean] cset.uuid 084505ad-5f6c-4666-9e3e-4f1650e8b015 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1796292 Title: Tight timeout for bcache removal causes spurious failures Status in curtin: Fix Released Status in linux package in Ubuntu: Confirmed Status in linux source package in Bionic: Confirmed Status in linux source package in Cosmic: Confirmed Status in linux source package in Disco: Confirmed Status in linux source package in Eoan: Confirmed Bug description: I've had a number of deployment faults where curtin would report Timeout exceeded for removal of /sys/fs/bcache/xxx when doing a mass- deployment of 30+ nodes. Upon retrying the node would usually deploy fine. Experimentally I've set the timeout ridiculously high, and it seems I'm getting no faults with this. I'm wondering if the timeout for removal is set too tight, or might need to be made configurable. --- curtin/util.py~ 2018-05-18 18:40:48.000000000 +0000 +++ curtin/util.py 2018-10-05 09:40:06.807390367 +0000 @@ -263,7 +263,7 @@ return _subp(*args, **kwargs) -def wait_for_removal(path, retries=[1, 3, 5, 7]): +def wait_for_removal(path, retries=[1, 3, 5, 7, 1200, 1200]): if not path: raise ValueError('wait_for_removal: missing path parameter') To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1796292/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
