You have been subscribed to a public bug:
Problem Description:
===================
'kdump-config show' shows Not ready to kdump even after doing kdump
setup on ubuntu160404 guest.
Steps to re-create:
==================
1. boslcp3 is up with BMC:1.20 & PNOR: 20180420 levels
2. Installed p8 compat guest with ubutnu160404
root@boslcp3g4:~# uname -a
Linux boslcp3g4 4.4.0-122-generic #146-Ubuntu SMP Mon Apr 23 15:33:25 UTC 2018
ppc64le ppc64le ppc64le GNU/Linux
root@boslcp3g4:~# uname -r
4.4.0-122-generic
3. Installed linux-crashdump crash dump utility.
4. root@boslcp3g4:~# cat /etc/default/grub.d/kexec-tools.cfg
GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT crashkernel=4096M"
root@boslcp3g4:~# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinux-4.4.0-116-generic
root=UUID=8be749b1-a5a8-4ab8-986c-ee4fcc61165e ro
root@boslcp3g4:~#
5. rebooted guest and checked kdump status.
root@boslcp3g4:~# /etc/init.d/kdump-tools status
? kdump-tools.service - Kernel crash dump capture service
Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor
preset: enabled)
Active: active (exited) since Sun 2018-05-06 04:18:37 EDT; 7min ago
Process: 2317 ExecStart=/etc/init.d/kdump-tools start (code=exited,
status=0/SUCCESS)
Main PID: 2317 (code=exited, status=0/SUCCESS)
Tasks: 0
Memory: 0B
CPU: 0
CGroup: /system.slice/kdump-tools.service
May 06 04:18:37 boslcp3g4 systemd[1]: Starting Kernel crash dump capture
service...
May 06 04:18:37 boslcp3g4 kdump-tools[2317]: Starting kdump-tools: * no
crashkernel= parameter in the kernel cmdline
May 06 04:18:37 boslcp3g4 systemd[1]: Started Kernel crash dump capture service.
root@boslcp3g4:~# sed -i 's/USE_KDUMP=0/USE_KDUMP=1/g' /etc/default/kdump-tools
4. kdump state
root@boslcp3g4:~# kdump-config show
* no crashkernel= parameter in the kernel cmdline
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr:
/var/lib/kdump/vmlinuz
kdump initrd:
/var/lib/kdump/initrd.img
current state: Not ready to kdump
kexec command:
no kexec command recorded
--> It still says 'Not ready to kdump'
--> Becuase of this state not able take dump when guest hangs/crashes
> Tried again changing crashkernel parameter on ubuntu160404 guest- p8
compat ,but still facing same issue
root@boslcp3g4:/kte/tools/setup.d# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinux-4.4.0-122-generic
root=UUID=8be749b1-a5a8-4ab8-986c-ee4fcc61165e ro ro splash quiet xmon=on
crashkernel=512M crashkernel=4096M
root@boslcp3g4:/kte/tools/setup.d#
root@boslcp3g4:/kte/tools/setup.d# /etc/init.d/kdump-tools status
? kdump-tools.service - Kernel crash dump capture service
Loaded: loaded (/lib/systemd/system/kdump-tools.service; enabled; vendor
preset: enabled)
Active: active (exited) since Sun 2018-05-06 04:59:51 EDT; 14min ago
Main PID: 2330 (code=exited, status=0/SUCCESS)
Tasks: 0
Memory: 0B
CPU: 0
CGroup: /system.slice/kdump-tools.service
May 06 04:59:50 boslcp3g4 systemd[1]: Starting Kernel crash dump capture se.....
May 06 04:59:51 boslcp3g4 kdump-tools[2330]: Starting kdump-tools: * Invali...g
May 06 04:59:51 boslcp3g4 kdump-tools[2330]: * Creating symlink /var/lib/kd...g
May 06 04:59:51 boslcp3g4 kdump-tools[2330]: * Invalid symlink : /var/lib/k...z
May 06 04:59:51 boslcp3g4 kdump-tools[2330]: * Creating symlink /var/lib/kd...z
May 06 04:59:51 boslcp3g4 kdump-tools[2330]: * /etc/default/kdump-tools: KD...g
May 06 04:59:51 boslcp3g4 systemd[1]: Started Kernel crash dump capture service.
Hint: Some lines were ellipsized, use -l to show in full.
root@boslcp3g4:/kte/tools/setup.d# kdump-config show
DUMP_MODE: kdump
USE_KDUMP: 1
KDUMP_SYSCTL: kernel.panic_on_oops=1
KDUMP_COREDIR: /var/crash
crashkernel addr:
/var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinux-4.4.0-122-generic
kdump initrd:
/var/lib/kdump/initrd.img: broken symbolic link to
/var/lib/kdump/initrd.img-4.4.0-122-generic
current state: Not ready to kdump
kexec command:
no kexec command recorded
--> Still kdump state shows 'Not ready to kdump'
Regards,
Indira
> > > Thanks Breno! You point out that 4.4.0-113 is in use. Where did you notice
> > > that? In the initial description, Indira states the guest has:
> > >
> > > root@boslcp3g4:~# uname -r
> > > 4.4.0-122-generic
> > >
> > > yet oddly /proc/cmdline shows:
> > >
> > > root@boslcp3g4:~# cat /proc/cmdline
> > > BOOT_IMAGE=/boot/vmlinux-4.4.0-116-generic
> > > root=UUID=8be749b1-a5a8-4ab8-986c-ee4fcc61165e ro
> >
> > That is where I see this -116 kernel. Is it -116 a symlink to -122?
>
> Hi Breno,
>
> I have installed ubtunu160404 using iso file which came up with
> '4.4.0-116-generic' And then did apt-get update/upgrade where kernel
> upgraded to '4.4.0-122-generic'
>
> Installed 'linux-crashdump' , updated crashkernel parameter in
> /etc/default/grub file & then issued reboot. Once system came up
> 'kdump-config show' command shows 'Not ready to dump'.
This is an interesting sequence that is probably not covered in kdump scripts.
Post installation scripts are triggered to build kdump initrd, in /var/lib/kdump
directory, for the booted kernel version while installing kdump-tools and for
every kernel installation after kdump-tools package is installed.
Since kernel version 4.4.0-122-generic was installed before kdump-tools was
installed and kdump-tools package was installed while the system was up
with 4.4.0-116-generic kernel version, kdump-tools postinstall script was
triggered only for kernel version 4.4.0-116-generic generating
/var/lib/kdump/initrd.img-4.4.0-116-generic
but NOT
/var/lib/kdump/initrd.img-4.4.0-122-generic
Subsequently, when system booted witt kernel version 4.4.0-122-generic,
/var/lib/kdump/initrd.img-4.4.0-122-generic was missing and AFAICS, kdump-tools
doesn't seem to have a mechanism to workaround this. I manually triggered
kdump-tools post install script on kernel version 4.4.0-122-generic
$ /etc/kernel/postinst.d/kdump-tools 4.4.0-122-generic
which ensured /var/lib/kdump/initrd.img-4.4.0-122-generic is created and was
able to load kdump kernel on 4.4.0-122-generic, after running the above
command with
$ systemctl restart kdump-tools.service
To workaround this problem:
A) Trigger kdump-tools post install script on every available kernel
while installing kdump-tools package.
B) Document the bevaior and provide the workaround commands.
C) Install kdump-tools as the basic package to avoid the issue altogether.
Please mirror the bug to Canonical to pick an appropriate approach to resolve
this problem.
Thanks
Hari
** Affects: makedumpfile (Ubuntu)
Importance: Undecided
Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
Status: New
** Tags: architecture-ppc64le bugnameltc-167483 severity-critical
targetmilestone-inin---
--
ISST-LTE:KVM:Ubuntu1804:BostonLC:boslcp3g4:ubuntu1604:'kdump-config show' shows
Not ready to kdump even after doing kdump setup.
https://bugs.launchpad.net/bugs/1769927
You received this bug notification because you are a member of Kernel Packages,
which is subscribed to makedumpfile in Ubuntu.
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
