Hi Chengen,
Thanks for the great work for these SRUs.
I'd like to suggest an update/improvement to the Test Plan section.
Even though crash is so broken that it can't even open a file,
once it starts working at all, it would be important to check
that it is working _correctly_.
So, please, could you add verifications for basic correctness
of the commands being addressed per patch (for GA and HWE)?
e.g., kmem -s|-S, module memory layout, etc.
And I had questions on 2 patches:
Patch 3:
---
31 + * commit e36ce448a08d removed kmem_cache.freelist_cache in 6.1,
32 + * so use freelist_size instead.
33 */
34 - if (MEMBER_EXISTS("kmem_cache", "freelist_cache")) {
35 + if (MEMBER_EXISTS("kmem_cache", "freelist_size")) {
This is an inconditional change before/after 6.1, which thus could
impact Jammy, as it ships both 5.15 and 6.2 kernels.
However, it seems the (new value) attribute 'freelist_size' already
exists, so it should be fine in Jammy _if_ 5.15 has it too.
Could you please confirm?
Patch 5:
---
Some of this backport's context update is because this patch is not included,
and it would also fit the 6.2 criteria (changes in 6.1). It seems only code
adds though, not sure how it changes any behavior (or more patches needed).
Can you clarify if it's not really needed for 6.5?
(I haven't followed the maple tree closely, but the commit message suggests
it's important.)
If it's not needed _right_ now, i.e., if this SRU is priority, and crash
at least _works_ (which is a good improvement), I think it would be fine
to add it later.
commit 872cad2d63b3a07f65323fe80a7abb29ea276b44
Author: Tao Liu <l...@redhat.com>
Date: Tue Jan 10 14:56:27 2023 +0800
Port the maple tree data structures and functions
There have been two ways to iterate vm_area_struct until Linux 6.0:
1) by rbtree, aka vma.vm_rb;
2) by linked list, aka vma.vm_{next,prev}.
However with the maple tree patches[1][2] in Linux 6.1, vm_rb and
vm_{next,prev} are removed from vm_area_struct. The vm_area_dump()
in crash mainly uses the linked list for vma iteration, which will
not work for this case. So the maple tree iteration needs to be
ported to crash.
This patch 5 is also big, adding a lot, anyway, but it goes to gdb-10.2.patch,
which only changed context lines for the backport, indeed.
A pure-code review against upstream seems a big effort (~2000 lines, I managed
up to ~1000, and it seems to match upstream).
I guess this patch will be reasonably verified if the crash commands to show
module memory layout on kernel 6.4+ (6.5 in our case) run correctly.
** Changed in: crash (Ubuntu)
Status: In Progress => Incomplete
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to crash in Ubuntu.
https://bugs.launchpad.net/bugs/2038249
Title:
The dump file parsing issue arises from structural changes in Linux
kernel 6.2
Status in crash package in Ubuntu:
Incomplete
Status in crash source package in Jammy:
In Progress
Status in crash source package in Lunar:
In Progress
Status in crash source package in Mantic:
In Progress
Bug description:
[Impact]
Linux kernel 6.2 includes patches with structural changes that may render the
crash utility unable to parse the dump file.
==========
d122019bf061 mm: Split slab into its own type
401fb12c68c2 mm: Differentiate struct slab fields by sl*b implementations
07f910f9b729 mm: Remove slab from struct page
0d9b1ffefabe arm64: mm: make vabits_actual a build time constant if possible
e36ce448a08d mm/slab: use kmalloc_node() for off slab freelist_idx_t array
allocation
130d4df57390 mm/sl[au]b: rearrange struct slab fields to allow larger rcu_head
ac3b43283923 module: replace module_layout with module_memory
b69f0aeb0689 pid: Replace struct pid 1-element array with flex-array
==========
[Fix]
It is advisable to adopt commits that address the structural changes issue.
==========
In 8.0.1:
- 14f8c460473c memory: Handle struct slab changes on Linux 5.17-rc1 and later
- 5f390ed811b0 Fix for "kmem -s|-S" and "bt -F[F]" on Linux 5.17-rc1
- b89f9ccf511a Fix for "kmem -s|-S" on Linux 5.17+ with CONFIG_SLAB
In 8.0.2:
- f02c8e87fccb arm64: use TCR_EL1_T1SZ to get the correct info if
vabits_actual is missing
In 8.0.3:
- d83df2fb66cd SLUB: Fix for offset change of struct slab members on Linux
6.2-rc1
- df1f0cba729f x86_64: Fix for move of per-cpu variables into struct pcpu_hot
- 120d6e89fc14 SLAB: Fix for "kmem -s|-S" options on Linux 6.1 and later
- ac96e17d1de5 SLAB: Fix for "kmem -s|-S" options on Linux 6.2-rc1 and later
In 8.0.3++ (8.0.4 development)
- 7750e61fdb2a Support module memory layout change on Linux 6.4
- 88580068b7dd Fix failure of gathering task table on Linux 6.5-rc1 and later
- 4ee56105881d Fix compilation error due to new strlcpy function that glibc
added
==========
[Test Plan]
1. Install the required packages and then proceed to reboot the machine.
# sudo apt install crash linux-crashdump -y
# reboot
2. To check the status of kdump, use the `kdump-config show` command.
# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x64000000
/var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-6.2.0-33-generic
kdump initrd:
/var/lib/kdump/initrd.img: symbolic link to
/var/lib/kdump/initrd.img-6.2.0-33-generic
current state: ready to kdump
kexec command:
/sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-6.2.0-33-generic
root=UUID=3e72f5d5-870b-4b8e-9a0d-8ba920391379 ro console=tty1 console=ttyS0
reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll
usbcore.nousb" --initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
3. To trigger a crash dump forcefully, execute the `echo c | sudo tee
/proc/sysrq-trigger` command.
4. Download the kernel .ddeb file, which will be used for analyzing the dump
file.
# sudo -i
# cd /var/crash
# pull-lp-ddebs linux-image-unsigned-$(uname -r)
# dpkg-deb -x linux-image-unsigned-$(uname -r)-*.ddeb dbgsym-$(uname -r)
5. Utilize the "crash" utility to parse and analyze the dump file.
crash 8.0.0
Copyright (C) 2002-2021 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005, 2011, 2020-2021 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
Copyright (C) 2015, 2021 VMware, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
WARNING: VA_BITS: calculated: 46 vmcoreinfo: 48
GNU gdb (GDB) 10.2
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-unknown-linux-gnu".
Type "show configuration" for configuration details.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
crash: seek error: kernel virtual address: ffffd59a92d48ae8 type: "possible"
WARNING: cannot read cpu_possible_map
crash: seek error: kernel virtual address: ffffd59a92d48b68 type: "present"
WARNING: cannot read cpu_present_map
crash: seek error: kernel virtual address: ffffd59a92d48aa8 type: "online"
WARNING: cannot read cpu_online_map
crash: seek error: kernel virtual address: ffffd59a92d48bb0 type: "active"
WARNING: cannot read cpu_active_map
crash: seek error: kernel virtual address: ffffd59a93288928 type:
"shadow_timekeeper xtime_sec"
crash: seek error: kernel virtual address: ffffd59a9317b8f0 type:
"init_uts_ns"
crash: dbgsym-6 and 202309251539/dump.202309251539 do not match!
Usage:
crash [OPTION]... NAMELIST MEMORY-IMAGE[@ADDRESS] (dumpfile form)
crash [OPTION]... [NAMELIST] (live system form)
Enter "crash -h" for details.
[Where problems could occur]
Significant structural changes have occurred between Linux kernel versions
5.15 and 6.2.
We are only incorporating patches to ensure the functionality of the "crash"
command.
However, please be aware that these patches will alter the parsing logic and
could potentially result in the "crash" utility being unable to parse the dump
file in the worst-case scenario.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/crash/+bug/2038249/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
