@Krister, no the affected series is related to where the primarily affected kernel version sits. For 5.15 kernels this is 22.04/Jammy. On AWS the custom kernels used, roll. So it is a 5.15 based kernel. 5.4 would be the native Focal kernel. But that is not affected. So this overall is not tracked against Focal.
** Description changed: + [SRU Justification] + + Impact: Some imbalanced ref-counting produces kernel warnings regularly. + Since it is a warning level, this triggers system monitoring on servers + which in turn causes unnecessary work for inspecting the logs. + + Fix: There is a fix upstream and also backported to the upstream stable + branch. However we are still a bit behind catching up with the latest + versions. Since this is having quite an impact and the fix is rather + straight forward, we pull this in from upstream stable ahead of time. + + Test case: tbd + + Regression potential: Regressions would manifest as different errors + related to ref-counting. + + --- + I have systems that are regularly hitting a WARN in trace_event_dyn_put_ref. The exact message is: WARNING: CPU: 1 PID: 30309 at kernel/trace/trace_dynevent.c:46 +trace_event_dyn_put_ref+0x15/0x20 With the following stacktrace: - perf_trace_init+0x8f/0xd0 - perf_tp_event_init+0x1f/0x40 - perf_try_init_event+0x4a/0x130 - perf_event_alloc+0x497/0xf40 - __do_sys_perf_event_open+0x1d4/0xf70 - __x64_sys_perf_event_open+0x20/0x30 - do_syscall_64+0x5c/0xc0 - entry_SYSCALL_64_after_hwframe+0x44/0xae + perf_trace_init+0x8f/0xd0 + perf_tp_event_init+0x1f/0x40 + perf_try_init_event+0x4a/0x130 + perf_event_alloc+0x497/0xf40 + __do_sys_perf_event_open+0x1d4/0xf70 + __x64_sys_perf_event_open+0x20/0x30 + do_syscall_64+0x5c/0xc0 + entry_SYSCALL_64_after_hwframe+0x44/0xae I've debugged this and worked with upstream to get a fix into Linux. It was recently merged in 6.0-rc2. See here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.0-rc2&id=7249921d94ff64f67b733eca0b68853a62032b3d The problem started appearing as soon as our systems picked up the linux-aws-5.15 branch for Focal. (That was 5.15.0-1015-aws, if memory serves). Could you please cherry pick this fix and pull it back to the the linux and linux-aws kernels for Focal? There's test here: https://lore.kernel.org/all/cover.1660347763.git.k...@templeofstupid.com/ that reproduces the problem very reliably for me. With the patch applied, I no longer get the WARNs. - --- + --- ProblemType: Bug AlsaDevices: - total 0 - crw-rw---- 1 root audio 116, 1 Aug 22 17:32 seq - crw-rw---- 1 root audio 116, 33 Aug 22 17:32 timer + total 0 + crw-rw---- 1 root audio 116, 1 Aug 22 17:32 seq + crw-rw---- 1 root audio 116, 33 Aug 22 17:32 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.11-0ubuntu27.24 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A CasperMD5CheckResult: skip DistroRelease: Ubuntu 20.04 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: Lsusb-t: - + Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1: MachineType: Amazon EC2 c5d.12xlarge Package: linux (not installed) PciMultimedia: - + ProcEnviron: - TERM=xterm-256color - PATH=(custom, no user) - LANG=C.UTF-8 - SHELL=/bin/bash + TERM=xterm-256color + PATH=(custom, no user) + LANG=C.UTF-8 + SHELL=/bin/bash ProcFB: - + ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-1015-aws root=PARTUUID=4986e35b-1bd5-45d3-b528-fa2edb861a38 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 panic=-1 ProcVersionSignature: Ubuntu 5.15.0-1015.19~20.04.1-aws 5.15.39 RelatedPackageVersions: - linux-restricted-modules-5.15.0-1015-aws N/A - linux-backports-modules-5.15.0-1015-aws N/A - linux-firmware N/A + linux-restricted-modules-5.15.0-1015-aws N/A + linux-backports-modules-5.15.0-1015-aws N/A + linux-firmware N/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' Tags: focal uec-images Uname: Linux 5.15.0-1015-aws x86_64 UnreportableReason: This report is about a package that is not installed. UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: N/A _MarkForUpload: False dmi.bios.date: 10/16/2017 dmi.bios.release: 1.0 dmi.bios.vendor: Amazon EC2 dmi.bios.version: 1.0 dmi.board.asset.tag: i-03f5d8581c7ad94aa dmi.board.vendor: Amazon EC2 dmi.chassis.asset.tag: Amazon EC2 dmi.chassis.type: 1 dmi.chassis.vendor: Amazon EC2 dmi.modalias: dmi:bvnAmazonEC2:bvr1.0:bd10/16/2017:br1.0:svnAmazonEC2:pnc5d.12xlarge:pvr:rvnAmazonEC2:rn:rvr:cvnAmazonEC2:ct1:cvr:sku: dmi.product.name: c5d.12xlarge dmi.sys.vendor: Amazon EC2 ** Changed in: linux (Ubuntu Jammy) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1987232 Title: WARN in trace_event_dyn_put_ref Status in linux package in Ubuntu: Confirmed Status in linux source package in Jammy: Fix Committed Status in linux source package in Kinetic: Confirmed Bug description: [SRU Justification] Impact: Some imbalanced ref-counting produces kernel warnings regularly. Since it is a warning level, this triggers system monitoring on servers which in turn causes unnecessary work for inspecting the logs. Fix: There is a fix upstream and also backported to the upstream stable branch. However we are still a bit behind catching up with the latest versions. Since this is having quite an impact and the fix is rather straight forward, we pull this in from upstream stable ahead of time. Test case: tbd Regression potential: Regressions would manifest as different errors related to ref-counting. --- I have systems that are regularly hitting a WARN in trace_event_dyn_put_ref. The exact message is: WARNING: CPU: 1 PID: 30309 at kernel/trace/trace_dynevent.c:46 +trace_event_dyn_put_ref+0x15/0x20 With the following stacktrace: perf_trace_init+0x8f/0xd0 perf_tp_event_init+0x1f/0x40 perf_try_init_event+0x4a/0x130 perf_event_alloc+0x497/0xf40 __do_sys_perf_event_open+0x1d4/0xf70 __x64_sys_perf_event_open+0x20/0x30 do_syscall_64+0x5c/0xc0 entry_SYSCALL_64_after_hwframe+0x44/0xae I've debugged this and worked with upstream to get a fix into Linux. It was recently merged in 6.0-rc2. See here: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v6.0-rc2&id=7249921d94ff64f67b733eca0b68853a62032b3d The problem started appearing as soon as our systems picked up the linux-aws-5.15 branch for Focal. (That was 5.15.0-1015-aws, if memory serves). Could you please cherry pick this fix and pull it back to the the linux and linux-aws kernels for Focal? There's test here: https://lore.kernel.org/all/cover.1660347763.git.k...@templeofstupid.com/ that reproduces the problem very reliably for me. With the patch applied, I no longer get the WARNs. --- ProblemType: Bug AlsaDevices: total 0 crw-rw---- 1 root audio 116, 1 Aug 22 17:32 seq crw-rw---- 1 root audio 116, 33 Aug 22 17:32 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.11-0ubuntu27.24 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CRDA: N/A CasperMD5CheckResult: skip DistroRelease: Ubuntu 20.04 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: Lsusb-t: Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1: MachineType: Amazon EC2 c5d.12xlarge Package: linux (not installed) PciMultimedia: ProcEnviron: TERM=xterm-256color PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash ProcFB: ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-1015-aws root=PARTUUID=4986e35b-1bd5-45d3-b528-fa2edb861a38 ro console=tty1 console=ttyS0 nvme_core.io_timeout=4294967295 panic=-1 ProcVersionSignature: Ubuntu 5.15.0-1015.19~20.04.1-aws 5.15.39 RelatedPackageVersions: linux-restricted-modules-5.15.0-1015-aws N/A linux-backports-modules-5.15.0-1015-aws N/A linux-firmware N/A RfKill: Error: [Errno 2] No such file or directory: 'rfkill' Tags: focal uec-images Uname: Linux 5.15.0-1015-aws x86_64 UnreportableReason: This report is about a package that is not installed. UpgradeStatus: No upgrade log present (probably fresh install) UserGroups: N/A _MarkForUpload: False dmi.bios.date: 10/16/2017 dmi.bios.release: 1.0 dmi.bios.vendor: Amazon EC2 dmi.bios.version: 1.0 dmi.board.asset.tag: i-03f5d8581c7ad94aa dmi.board.vendor: Amazon EC2 dmi.chassis.asset.tag: Amazon EC2 dmi.chassis.type: 1 dmi.chassis.vendor: Amazon EC2 dmi.modalias: dmi:bvnAmazonEC2:bvr1.0:bd10/16/2017:br1.0:svnAmazonEC2:pnc5d.12xlarge:pvr:rvnAmazonEC2:rn:rvr:cvnAmazonEC2:ct1:cvr:sku: dmi.product.name: c5d.12xlarge dmi.sys.vendor: Amazon EC2 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1987232/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
