Default Comment by Bridge ** Attachment added: "dmesg message logs during hotplug cpu" https://bugs.launchpad.net/bugs/1759722/+attachment/5094238/+files/hotplug_attachcpu
** Changed in: ubuntu Assignee: (unassigned) => Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) ** Package changed: ubuntu => linux (Ubuntu) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1759722 Title: ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3: cpu hotplug on boslcp3g4 guest dumping call traces continuously. Status in linux package in Ubuntu: New Bug description: Problem Description: =================== After performing HOTPLUG CPU on guest boslcp3g4 , saw dumping messages on the guest console continously Steps to re-create: ================== > boslcp3g4 is a guest of Boston LC system "boslcp3". Both of these systems are installed with ubuntu1804 kernel. root@boslcp3:/home# uname -a Linux boslcp3 4.15.0-12-generic #13+leo20180320 SMP Tue Mar 20 13:10:42 CDT 2018 ppc64le ppc64le ppc64le GNU/Linux root@boslcp3:/home# uname -r 4.15.0-12-generic root@boslcp3g4:/kte/tools/setup.d# uname -a Linux boslcp3g4 4.15.0-12-generic #13+leo20180320 SMP Tue Mar 20 13:10:42 CDT 2018 ppc64le ppc64le ppc64le GNU/Linux root@boslcp3g4:/kte/tools/setup.d# uname -r 4.15.0-12-generic > When I perform hotplug of the cpu , it was successful but throwing these call traces in the Guest console continuously- root@boslcp3:~# virsh setvcpus boslcp3g4 64 --live root@boslcp3g4:~# [22273.371233] Using 'aes-generic' as fallback implementation. [22273.371377] Using 'ctr(aes-generic)' as fallback implementation. [22273.450867] Using 'aes-generic' as fallback implementation. [22273.450967] Using 'ctr(aes-generic)' as fallback implementation. [22300.779703] Using 'aes-generic' as fallback implementation. [22300.779810] Using 'ctr(aes-generic)' as fallback implementation. [22300.789192] Using 'aes-generic' as fallback implementation. [22300.789327] Using 'ctr(aes-generic)' as fallback implementation. [22327.714322] Using 'aes-generic' as fallback implementation. [22327.714423] Using 'ctr(aes-generic)' as fallback implementation. [22327.722548] Using 'aes-generic' as fallback implementation. [22327.722654] Using 'ctr(aes-generic)' as fallback implementation. [22354.401415] Using 'aes-generic' as fallback implementation. [22354.401546] Using 'ctr(aes-generic)' as fallback implementation. [22354.422227] Using 'aes-generic' as fallback implementation. [22354.422354] Using 'ctr(aes-generic)' as fallback implementation. [22368.150663] WARNING: CPU: 34 PID: 40114 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.153463] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.154166] CPU: 34 PID: 40114 Comm: kworker/34:0H Not tainted 4.15.0-12-generic #13+leo20180320 [22368.154241] Workqueue: kblockd blk_mq_run_work_fn [22368.154283] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.154344] REGS: c000000145a2f9d0 TRAP: 0700 Not tainted (4.15.0-12-generic) [22368.154402] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.154480] CFAR: c00000000068e070 SOFTE: 1 [22368.154480] GPR00: c000000000131538 c000000145a2fc50 c0000000016eaf00 c0000001346f7800 [22368.154480] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8 [22368.154480] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000145985d30 [22368.154480] GPR12: c00000000068e140 c00000000fa97600 c00000000013a368 c0000001760c35c0 [22368.154480] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.154480] GPR20: c00000017ee78180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.154480] GPR24: 0000000000000402 0000000000000000 c00000017ee78500 0000000000000000 [22368.154480] GPR28: c0000001346f7880 c00000017ee7c900 c00000017ee78180 c0000001346f7800 [22368.155001] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.155065] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.155115] Call Trace: [22368.155139] [c000000145a2fc50] [c000000145a2fce0] 0xc000000145a2fce0 (unreliable) [22368.155200] [c000000145a2fc90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.155259] [c000000145a2fd20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.155308] [c000000145a2fdc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.155358] [c000000145a2fe30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.155415] Instruction dump: [22368.155446] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.155506] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.155566] ---[ end trace 70e3c0daa532c721 ]--- [22368.204754] WARNING: CPU: 34 PID: 40224 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.204856] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.205275] CPU: 34 PID: 40224 Comm: kworker/34:1H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.205352] Workqueue: kblockd blk_mq_run_work_fn [22368.205392] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.205449] REGS: c00000017ac8b9d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.205513] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 44002284 XER: 00000000 [22368.205588] CFAR: c00000000068e070 SOFTE: 1 [22368.205588] GPR00: c000000000131538 c00000017ac8bc50 c0000000016eaf00 c0000001346f7800 [22368.205588] GPR04: c0000001346f7880 0000000000000000 0000000044008282 c000000001721ed8 [22368.205588] GPR08: 0000000000000000 0000000000000001 0000000000000000 00000000ffffffff [22368.205588] GPR12: c00000000068e140 c00000000fa97600 c00000000013a368 c000000179360000 [22368.205588] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.205588] GPR20: c00000017ee78180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.205588] GPR24: 0000000000000402 0000000000000000 c00000017ee78500 0000000000000000 [22368.205588] GPR28: c0000001346f7880 c00000017ee7c900 c00000017ee78180 c0000001346f7800 [22368.206068] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.206118] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.206165] Call Trace: [22368.206263] [c00000017ac8bc90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.206322] [c00000017ac8bd20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.206373] [c00000017ac8bdc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.206424] [c00000017ac8be30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.206480] Instruction dump: [22368.206511] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.206588] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.206662] ---[ end trace 70e3c0daa532c722 ]--- [22368.340886] WARNING: workqueue cpumask: online intersect > possible intersect [22368.378115] WARNING: CPU: 36 PID: 40286 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.378224] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.378684] CPU: 36 PID: 40286 Comm: kworker/36:0H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.378762] Workqueue: kblockd blk_mq_run_work_fn [22368.378801] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.378859] REGS: c00000017ace79d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.378942] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.379015] CFAR: c00000000068e070 SOFTE: 1 [22368.379015] GPR00: c000000000131538 c00000017ace7c50 c0000000016eaf00 c000000142f0fc00 [22368.379015] GPR04: c000000142f0fc80 0000000000000000 0000000000000000 c000000001721ed8 [22368.379015] GPR08: 0000000000000000 0000000000000001 0000000000000000 c0000001788e2930 [22368.379015] GPR12: c00000000068e140 c00000000fa98c00 c00000000013a368 c0000001760c2400 [22368.379015] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.379015] GPR20: c00000017eef8180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.379015] GPR24: 0000000000000402 0000000000000000 c00000017eef8500 0000000000000000 [22368.379015] GPR28: c000000142f0fc80 c00000017eefc900 c00000017eef8180 c000000142f0fc00 [22368.379497] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.379547] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.379594] Call Trace: [22368.379617] [c00000017ace7c50] [c00000017ace7ce0] 0xc00000017ace7ce0 (unreliable) [22368.379678] [c00000017ace7c90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.379739] [c00000017ace7d20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.379790] [c00000017ace7dc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.379842] [c00000017ace7e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.379907] Instruction dump: [22368.379938] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.380005] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.380070] ---[ end trace 70e3c0daa532c723 ]--- [22368.488613] WARNING: CPU: 39 PID: 40552 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.488724] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.489289] CPU: 39 PID: 40552 Comm: kworker/39:0H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.489367] Workqueue: kblockd blk_mq_run_work_fn [22368.489409] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.489467] REGS: c00000017285b9d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.489532] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.489605] CFAR: c00000000068e070 SOFTE: 1 [22368.489605] GPR00: c000000000131538 c00000017285bc50 c0000000016eaf00 c0000001346f7800 [22368.489605] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8 [22368.489605] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000142ca9c30 [22368.489605] GPR12: c00000000068e140 c00000000fa9ad00 c00000000013a368 c0000001760c9e40 [22368.489605] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.489605] GPR20: c00000017efb8180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.489605] GPR24: 0000000000000402 0000000000000000 c00000017efb8500 0000000000000000 [22368.489605] GPR28: c0000001346f7880 c00000017efbc900 c00000017efb8180 c0000001346f7800 [22368.490093] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.490144] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.490191] Call Trace: [22368.490213] [c00000017285bc50] [c00000017285bce0] 0xc00000017285bce0 (unreliable) [22368.490272] [c00000017285bc90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.490330] [c00000017285bd20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.490379] [c00000017285bdc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.490431] [c00000017285be30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.490487] Instruction dump: [22368.490517] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.490576] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.490636] ---[ end trace 70e3c0daa532c724 ]--- [22368.522251] WARNING: CPU: 39 PID: 41050 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.522353] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.522772] CPU: 39 PID: 41050 Comm: kworker/39:1H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.522851] Workqueue: kblockd blk_mq_run_work_fn [22368.522890] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.522962] REGS: c0000000e95e39d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.523029] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 44002884 XER: 00000000 [22368.523103] CFAR: c00000000068e070 SOFTE: 1 [22368.523103] GPR00: c000000000131538 c0000000e95e3c50 c0000000016eaf00 c0000001346f7800 [22368.523103] GPR04: c0000001346f7880 0000000000000000 0000000044008882 c000000001721ed8 [22368.523103] GPR08: 0000000000000000 0000000000000001 0000000000000000 00000000ffffffff [22368.523103] GPR12: c00000000068e140 c00000000fa9ad00 c00000000013a368 c000000261facac0 [22368.523103] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.523103] GPR20: c00000017efb8180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.523103] GPR24: 0000000000000402 0000000000000000 c00000017efb8500 0000000000000000 [22368.523103] GPR28: c0000001346f7880 c00000017efbc900 c00000017efb8180 c0000001346f7800 [22368.523584] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.523635] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.523682] Call Trace: [22368.523704] [c0000000e95e3c90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.523761] [c0000000e95e3d20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.523811] [c0000000e95e3dc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.523862] [c0000000e95e3e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.523919] Instruction dump: [22368.523948] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.524009] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.524068] ---[ end trace 70e3c0daa532c725 ]--- [22368.525105] WARNING: CPU: 38 PID: 40474 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.525198] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.525604] CPU: 38 PID: 40474 Comm: kworker/38:0H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.525692] Workqueue: kblockd blk_mq_run_work_fn [22368.525736] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.525800] REGS: c00000017282f9d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.525869] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.525944] CFAR: c00000000068e070 SOFTE: 1 [22368.525944] GPR00: c000000000131538 c00000017282fc50 c0000000016eaf00 c000000142f0fc00 [22368.525944] GPR04: c000000142f0fc80 0000000000000000 0000000000000000 c000000001721ed8 [22368.525944] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000142ca5730 [22368.525944] GPR12: c00000000068e140 c00000000fa9a200 c00000000013a368 c0000001760c4040 [22368.525944] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.525944] GPR20: c00000017ef78180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.525944] GPR24: 0000000000000402 0000000000000000 c00000017ef78500 0000000000000000 [22368.525944] GPR28: c000000142f0fc80 c00000017ef7c900 c00000017ef78180 c000000142f0fc00 [22368.526428] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.526477] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.526524] Call Trace: [22368.526546] [c00000017282fc50] [c00000017282fce0] 0xc00000017282fce0 (unreliable) [22368.526604] [c00000017282fc90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.526662] [c00000017282fd20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.526711] [c00000017282fdc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.526761] [c00000017282fe30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.526818] Instruction dump: [22368.526849] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.526908] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.526984] ---[ end trace 70e3c0daa532c726 ]--- [22368.829929] WARNING: CPU: 32 PID: 39931 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.830040] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.830473] CPU: 32 PID: 39931 Comm: kworker/32:0H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.830588] Workqueue: kblockd blk_mq_run_work_fn [22368.830629] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.830688] REGS: c0000001789779d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.830754] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.830830] CFAR: c00000000068e070 SOFTE: 1 [22368.830830] GPR00: c000000000131538 c000000178977c50 c0000000016eaf00 c0000001346f7800 [22368.830830] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8 [22368.830830] GPR08: 0000000000000000 0000000000000001 0000000000000000 c0000001727a5730 [22368.830830] GPR12: c00000000068e140 c00000000fa96000 c00000000013a368 c0000001760ca940 [22368.830830] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.830830] GPR20: c00000017edf8180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.830830] GPR24: 0000000000000402 0000000000000000 c00000017edf8500 0000000000000000 [22368.830830] GPR28: c0000001346f7880 c00000017edfc900 c00000017edf8180 c0000001346f7800 [22368.831330] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.831383] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.831472] Call Trace: [22368.831495] [c000000178977c50] [c000000178977ce0] 0xc000000178977ce0 (unreliable) [22368.831554] [c000000178977c90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.831612] [c000000178977d20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.831662] [c000000178977dc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.831714] [c000000178977e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.831773] Instruction dump: [22368.831803] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.831863] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.831924] ---[ end trace 70e3c0daa532c727 ]--- [22368.831972] WARNING: CPU: 32 PID: 39931 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.832047] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.832450] CPU: 32 PID: 39931 Comm: kworker/32:0H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.832527] Workqueue: kblockd blk_mq_run_work_fn [22368.832565] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.832624] REGS: c0000001789779d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.832690] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.832761] CFAR: c00000000068e070 SOFTE: 1 [22368.832761] GPR00: c000000000131538 c000000178977c50 c0000000016eaf00 c0000001346f7800 [22368.832761] GPR04: c0000001346f7880 0000000000000000 0000000000000000 c000000001721ed8 [22368.832761] GPR08: 0000000000000000 0000000000000001 0000000000000000 c0000001727a5730 [22368.832761] GPR12: c00000000068e140 c00000000fa96000 c00000000013a368 c0000001760ca940 [22368.832761] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.832761] GPR20: c00000017edf8180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.832761] GPR24: 0000000000000402 0000000000000000 c00000017edf8500 0000000000000000 [22368.832761] GPR28: c0000001346f7880 c00000017edfc900 c00000017edf8180 c0000001346f7800 [22368.833253] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.833302] LR [c000000000131538] process_one_work+0x298/0x5a0 [22368.833351] Call Trace: [22368.833371] [c000000178977c50] [c000000178977c90] 0xc000000178977c90 (unreliable) [22368.833429] [c000000178977c90] [c000000000131538] process_one_work+0x298/0x5a0 [22368.833488] [c000000178977d20] [c0000000001318d8] worker_thread+0x98/0x630 [22368.833537] [c000000178977dc0] [c00000000013a508] kthread+0x1a8/0x1b0 [22368.833587] [c000000178977e30] [c00000000000b528] ret_from_kernel_thread+0x5c/0xb4 [22368.833642] Instruction dump: [22368.833673] 7fa4eb78 7fc3f378 4bb10611 60000000 eba10028 ebc10030 38210040 e8010010 [22368.833732] ebe1fff8 7c0803a6 4e800020 60420000 <0fe00000> 78290464 8129000c 552902ef [22368.833792] ---[ end trace 70e3c0daa532c728 ]--- [22368.913232] WARNING: CPU: 35 PID: 40184 at /home/user/ubuntu-bionic/block/blk-mq.c:1211 __blk_mq_run_hw_queue+0xf0/0x140 [22368.913335] Modules linked in: sctp esp4 xfrm4_mode_transport ccm xfrm_user xfrm_algo minix nls_iso8859_1 xfs rpcsec_gss_krb5 nfsv4 nfs fscache binfmt_misc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp nfsd libiscsi scsi_transport_iscsi auth_rpcgss parport_pc nfs_acl ppdev lockd grace sunrpc lp parport ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear crct10dif_vpmsum crc32c_vpmsum virtio_blk virtio_scsi virtio_net [22368.913750] CPU: 35 PID: 40184 Comm: kworker/35:0H Tainted: G W 4.15.0-12-generic #13+leo20180320 [22368.913830] Workqueue: kblockd blk_mq_run_work_fn [22368.913869] NIP: c00000000068e0f0 LR: c000000000131538 CTR: c00000000068e140 [22368.913928] REGS: c00000017acc79d0 TRAP: 0700 Tainted: G W (4.15.0-12-generic) [22368.913996] MSR: 800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE> CR: 42002424 XER: 00000000 [22368.914070] CFAR: c00000000068e070 SOFTE: 1 [22368.914070] GPR00: c000000000131538 c00000017acc7c50 c0000000016eaf00 c000000142f0fc00 [22368.914070] GPR04: c000000142f0fc80 0000000000000000 0000000000000000 c000000001721ed8 [22368.914070] GPR08: 0000000000000000 0000000000000001 0000000000000000 c000000172428530 [22368.914070] GPR12: c00000000068e140 c00000000fa98100 c00000000013a368 c0000001760c2ac0 [22368.914070] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 [22368.914070] GPR20: c00000017eeb8180 0000000000000000 0000000000000000 fffffffffffffef7 [22368.914070] GPR24: 0000000000000402 0000000000000000 c00000017eeb8500 0000000000000000 [22368.914070] GPR28: c000000142f0fc80 c00000017eebc900 c00000017eeb8180 c000000142f0fc00 [22368.914583] NIP [c00000000068e0f0] __blk_mq_run_hw_queue+0xf0/0x140 [22368.914635] LR [c000000000131538] process_one_work+0x298/0x5a0 [ > virsh dumpxml boslcp3g4 root@boslcp3:~# virsh dumpxml boslcp3g4 | grep vcpu <vcpu placement='static' current='64'>80</vcpu> <vcpus> <vcpu id='0' enabled='yes' hotpluggable='no' order='1'/> <vcpu id='1' enabled='yes' hotpluggable='no' order='1'/> <vcpu id='2' enabled='yes' hotpluggable='no' order='1'/> <vcpu id='3' enabled='yes' hotpluggable='no' order='1'/> <vcpu id='4' enabled='yes' hotpluggable='no' order='2'/> <vcpu id='5' enabled='yes' hotpluggable='no' order='2'/> <vcpu id='6' enabled='yes' hotpluggable='no' order='2'/> <vcpu id='7' enabled='yes' hotpluggable='no' order='2'/> <vcpu id='8' enabled='yes' hotpluggable='no' order='3'/> <vcpu id='9' enabled='yes' hotpluggable='no' order='3'/> <vcpu id='10' enabled='yes' hotpluggable='no' order='3'/> <vcpu id='11' enabled='yes' hotpluggable='no' order='3'/> <vcpu id='12' enabled='yes' hotpluggable='no' order='4'/> <vcpu id='13' enabled='yes' hotpluggable='no' order='4'/> <vcpu id='14' enabled='yes' hotpluggable='no' order='4'/> <vcpu id='15' enabled='yes' hotpluggable='no' order='4'/> <vcpu id='16' enabled='yes' hotpluggable='no' order='5'/> <vcpu id='17' enabled='yes' hotpluggable='no' order='5'/> <vcpu id='18' enabled='yes' hotpluggable='no' order='5'/> <vcpu id='19' enabled='yes' hotpluggable='no' order='5'/> <vcpu id='20' enabled='yes' hotpluggable='no' order='6'/> <vcpu id='21' enabled='yes' hotpluggable='no' order='6'/> <vcpu id='22' enabled='yes' hotpluggable='no' order='6'/> <vcpu id='23' enabled='yes' hotpluggable='no' order='6'/> <vcpu id='24' enabled='yes' hotpluggable='no' order='7'/> <vcpu id='25' enabled='yes' hotpluggable='no' order='7'/> <vcpu id='26' enabled='yes' hotpluggable='no' order='7'/> <vcpu id='27' enabled='yes' hotpluggable='no' order='7'/> <vcpu id='28' enabled='yes' hotpluggable='no' order='8'/> <vcpu id='29' enabled='yes' hotpluggable='no' order='8'/> <vcpu id='30' enabled='yes' hotpluggable='no' order='8'/> <vcpu id='31' enabled='yes' hotpluggable='no' order='8'/> <vcpu id='32' enabled='yes' hotpluggable='yes' order='9'/> <vcpu id='33' enabled='yes' hotpluggable='yes' order='9'/> <vcpu id='34' enabled='yes' hotpluggable='yes' order='9'/> <vcpu id='35' enabled='yes' hotpluggable='yes' order='9'/> <vcpu id='36' enabled='yes' hotpluggable='yes' order='10'/> <vcpu id='37' enabled='yes' hotpluggable='yes' order='10'/> <vcpu id='38' enabled='yes' hotpluggable='yes' order='10'/> <vcpu id='39' enabled='yes' hotpluggable='yes' order='10'/> <vcpu id='40' enabled='yes' hotpluggable='yes' order='11'/> <vcpu id='41' enabled='yes' hotpluggable='yes' order='11'/> <vcpu id='42' enabled='yes' hotpluggable='yes' order='11'/> <vcpu id='43' enabled='yes' hotpluggable='yes' order='11'/> <vcpu id='44' enabled='yes' hotpluggable='yes' order='12'/> <vcpu id='45' enabled='yes' hotpluggable='yes' order='12'/> <vcpu id='46' enabled='yes' hotpluggable='yes' order='12'/> <vcpu id='47' enabled='yes' hotpluggable='yes' order='12'/> <vcpu id='48' enabled='yes' hotpluggable='yes' order='13'/> <vcpu id='49' enabled='yes' hotpluggable='yes' order='13'/> <vcpu id='50' enabled='yes' hotpluggable='yes' order='13'/> <vcpu id='51' enabled='yes' hotpluggable='yes' order='13'/> <vcpu id='52' enabled='yes' hotpluggable='yes' order='14'/> <vcpu id='53' enabled='yes' hotpluggable='yes' order='14'/> <vcpu id='54' enabled='yes' hotpluggable='yes' order='14'/> <vcpu id='55' enabled='yes' hotpluggable='yes' order='14'/> <vcpu id='56' enabled='yes' hotpluggable='yes' order='15'/> <vcpu id='57' enabled='yes' hotpluggable='yes' order='15'/> <vcpu id='58' enabled='yes' hotpluggable='yes' order='15'/> <vcpu id='59' enabled='yes' hotpluggable='yes' order='15'/> <vcpu id='60' enabled='yes' hotpluggable='yes' order='16'/> <vcpu id='61' enabled='yes' hotpluggable='yes' order='16'/> <vcpu id='62' enabled='yes' hotpluggable='yes' order='16'/> <vcpu id='63' enabled='yes' hotpluggable='yes' order='16'/> <vcpu id='64' enabled='no' hotpluggable='yes'/> <vcpu id='65' enabled='no' hotpluggable='yes'/> <vcpu id='66' enabled='no' hotpluggable='yes'/> <vcpu id='67' enabled='no' hotpluggable='yes'/> <vcpu id='68' enabled='no' hotpluggable='yes'/> <vcpu id='69' enabled='no' hotpluggable='yes'/> <vcpu id='70' enabled='no' hotpluggable='yes'/> <vcpu id='71' enabled='no' hotpluggable='yes'/> <vcpu id='72' enabled='no' hotpluggable='yes'/> <vcpu id='73' enabled='no' hotpluggable='yes'/> <vcpu id='74' enabled='no' hotpluggable='yes'/> <vcpu id='75' enabled='no' hotpluggable='yes'/> <vcpu id='76' enabled='no' hotpluggable='yes'/> <vcpu id='77' enabled='no' hotpluggable='yes'/> <vcpu id='78' enabled='no' hotpluggable='yes'/> <vcpu id='79' enabled='no' hotpluggable='yes'/> </vcpus> root@boslcp3:~# > lspcu output from guest root@boslcp3g4:~# lscpu Architecture: ppc64le Byte Order: Little Endian CPU(s): 64 On-line CPU(s) list: 0,4,8,12,16,20,24,28,32-63 Off-line CPU(s) list: 1-3,5-7,9-11,13-15,17-19,21-23,25-27,29-31 Thread(s) per core: 2 Core(s) per socket: 8 Socket(s): 2 NUMA node(s): 2 Model: 2.2 (pvr 004e 1202) Model name: POWER9 (architected), altivec supported Hypervisor vendor: KVM Virtualization type: para L1d cache: 32K L1i cache: 32K NUMA node0 CPU(s): 0,4,8,12,16,20,24,28,32-39 NUMA node1 CPU(s): 40-63 root@boslcp3g4:~# > Attched guest consoel logs Hi, there is a patch to remove the WARNING message but it's harmless according to the patch author: 4058ba173eb411422482125e14e7bb12b70c8b38 blk-mq: turn WARN_ON in __blk_mq_run_hw_queue into printk We know this WARN_ON is harmless and in reality it may be trigged, so convert it to printk() and dump_stack() to avoid to confusing people. I'll prepare a backport to Canonical but I lowering this importance to medium, since it's not a problem. Thank you To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1759722/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp