### VERIFICATION-FAILED FOCAL-PROPOSED ###
makedumpfile original version: 1:1.6.7-1ubuntu2.4
makedumpfile proposed version: 1:1.6.7-1ubuntu2.5
kernel version: 5.15.0-89-generic hwe
sudo apt install -y linux-crashdump
sudo vim /etc/default/grub.d/kdump-tools.cfg
# Add the line at the top of the file below USE_KDUMP=1
LOAD_KEXEC=true
# Uncomment the makedumpfile line and change 31 to 32
MAKEDUMP_ARGS="-c -d 32"
exit vim
sudo vim /etc/default/grub.d/kdump-tools.cfg
change 192 to either 256M or 512M
sudo sysctl -w kernel.sysrq=1
sudo apt-get update-grub
sudo reboot
sudo su
kdump-config show
Needs to look similar to:
root@focal:/home/ubuntu# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr: 0x73000000
/var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-5.15.0-89-generic
kdump initrd:
/var/lib/kdump/initrd.img: symbolic link to
/var/lib/kdump/initrd.img-5.15.0-89-generic
current state: ready to kdump
kexec command:
/sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.15.0-89-generic
root=UUID=975b9a95-b58e-48da-bd23-dd01b13bcbad ro quiet splash vt.handoff=7
reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb"
--initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
*most importantly the addr is not zero and the state is: ready to kdump.
# trigger crash
echo c > /proc/sysrq-trigger
... wait for reboot and login
cd /var/crash/
There is a folder with a datetimestamp of the crash, inside the folder is the
vmcore and dmesg files
There is also a file called linux-image*.crash in /var/crash/
*Note: In bionic we get the cp message
[ 6.946187] kdump-tools[622]: Starting kdump-tools: * running makedumpfile
-c -d 32 /proc/vmcore /var/crash/202311221422/dump-incomplete
[ 6.959932] kdump-tools[622]: Dump_level(32) is invalid.
[ 6.964316] kdump-tools[622]: makedumpfile Failed.
[ 6.976231] kdump-tools[622]: * kdump-tools: makedumpfile failed, falling
back to 'cp'
[ 25.084729] kdump-tools[622]: * kdump-tools: saved vmcore in
/var/crash/202311221422
[ 26.355039] kdump-tools[622]: * running makedumpfile --dump-dmesg
/proc/vmcore /var/crash/202311221422/dmesg.202311221422
[ 26.436513] kdump-tools[622]: The dmesg log is saved to
/var/crash/202311221422/dmesg.202311221422.
[ 26.443208] kdump-tools[622]: makedumpfile Completed.
[ 26.449066] kdump-tools[622]: * kdump-tools: saved dmesg content in
/var/crash/202311221422
[ 26.463950] kdump-tools[622]: Wed, 22 Nov 2023 14:22:50 +0000
[ 26.490318] kdump-tools[622]: Rebooting.
*Note: In Focal we don't
Nov 28 13:35:54 focal kdump-tools[604]: Starting kdump-tools:
Nov 28 13:35:54 focal kdump-tools[667]: Starting kdump-tools:
Nov 28 13:35:54 focal kdump-tools[667]: * Creating symlink /
Nov 28 13:35:54 focal kdump-tools[708]: * Creating symlink /
Nov 28 13:35:54 focal kdump-tools[708]: n: failed to create symbolic link '/va:
No such file or directory
Nov 28 13:35:54 focal kdump-tools[713]: kdump-tools: Generating
/var/lib/kdump/initrd.img-5.15.0-89-generic
Nov 28 13:35:57 focal kdump-tools[667]: * Creating symlink
/var/lib/kdump/initrd.img
Nov 28 13:35:57 focal kdump-tools[667]: * Invalid symlink :
/var/lib/kdump/vmlinuz
Nov 28 13:35:57 focal kdump-tools[667]: * Creating symlink
/var/lib/kdump/vmlinuz
Nov 28 13:35:58 focal kdump-tools[667]: * loaded kdump kernel
Nov 28 13:35:58 focal kdump-tools[3337]: /sbin/kexec -p
--command-line="BOOT_IMAGE=/boot/vmlinuz-5.15.0-89-generic
root=UUID=975b9a95-b58e-48da-bd23-dd01b13bcbad ro quiet splash vt.handoff=7
reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb"
--initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
Nov 28 13:35:58 focal kdump-tools[3338]: loaded kdump kernel
-------
enabled proposed repo
Remove any crashes from /var/crash/
install proposed version of makedumpfile
"follow similar steps above if on new machine"
/var/log/syslog
Nov 28 15:10:19 focal kdump-tools[674]: Starting kdump-tools:
Nov 28 15:10:19 focal kdump-tools[674]: * Creating symlink / * Creating symlink
/var/lib/kdump/initrd.img
Nov 28 15:10:20 focal kdump-tools[674]: * loaded kdump kernel
Nov 28 15:10:20 focal kdump-tools: /sbin/kexec -p
--command-line="BOOT_IMAGE=/boot/vmlinuz-5.15.0-89-generic
root=UUID=1887d6a1-cdd3-474f-a3a8-53703a658358 ro quiet splash vt.handoff=7
reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb"
--initrd=/var/lib/kdump/initrd.img /var/lib/kdump/vmlinuz
Nov 28 15:10:20 focal kdump-tools: loaded kdump kernel
You can produce a vmcore 2 ways.
1. by editing the file /etc/default/kdump-tools to
commenting in MAKEDUMPFILE_ARGS="-c -d 32"
2. by command
mv /usr/bin/makedumpfile /usr/bin/makedumpfile.bak
My results show that with no changes to cause a crash, we see a dmesg and dump
files created in the /var/crash/timestamp folder which is the correct workflow.
However, purposely causing a crash to happen results in a vmcore the size of
the installed RAM.
Cheers,
Heather Lemon
*If this needs to be re-tested, please make a note and explanation.
### VERIFICATION-FAILED FOCAL-PROPOSED ###
** Tags removed: verification-needed-focal
** Tags added: verification-failed-focal
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to makedumpfile in Ubuntu.
https://bugs.launchpad.net/bugs/1970672
Title:
makedumpfile falls back to cp with "__vtop4_x86_64: Can't get a valid
pmd_pte."
Status in makedumpfile package in Ubuntu:
Fix Released
Status in makedumpfile source package in Focal:
Fix Committed
Bug description:
[Impact]
* On Focal with an HWE (>=5.12) kernel, makedumpfile can sometimes fail with
"__vtop4_x86_64: Can't get a valid pmd_pte."
* makedumpfile falls back to cp for the dump, resulting in extremely
large vmcores. This can impact both collection and analysis due to
lack of space for the resulting vmcore.
* This is fixed in upstream commit present in versions 1.7.0 and 1.7.1:
https://github.com/makedumpfile/makedumpfile/commit/646456862df8926ba10dd7330abf3bf0f887e1b6
commit 646456862df8926ba10dd7330abf3bf0f887e1b6
Author: Kazuhito Hagio <k-hagio...@nec.com>
Date: Wed May 26 14:31:26 2021 +0900
[PATCH] Increase SECTION_MAP_LAST_BIT to 5
* Required for kernel 5.12
Kernel commit 1f90a3477df3 ("mm: teach pfn_to_online_page() about
ZONE_DEVICE section collisions") added a section flag
(SECTION_TAINT_ZONE_DEVICE) and causes makedumpfile an error on
some machines like this:
__vtop4_x86_64: Can't get a valid pmd_pte.
readmem: Can't convert a virtual address(ffffe2bdc2000000) to physical
address.
readmem: type_addr: 0, addr:ffffe2bdc2000000, size:32768
__exclude_unnecessary_pages: Can't read the buffer of struct page.
create_2nd_bitmap: Can't exclude unnecessary pages.
Increase SECTION_MAP_LAST_BIT to 5 to fix this. The bit had not
been used until the change, so we can just increase the value.
Signed-off-by: Kazuhito Hagio <k-hagio...@nec.com>
[Test Plan]
* Confirm that makedumpfile works as expected by triggering a kdump.
* Confirm that the patched makedumpfile works as expected on a system
known to experience the issue.
* Confirm that the patched makedumpfile is able to work with a cp-
generated known affected vmcore to compress it. The unpatched version
fails.
[Where problems could occur]
* This change could adversely affect the collection/compression of
vmcores during a kdump situation resulting in fallback to cp.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/makedumpfile/+bug/1970672/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
