------- Comment From hbath...@in.ibm.com 2018-09-04 05:12 EDT------- crash tool version in proposed works fien with 4.15.0-33-generic kernel:
-- root@ltc-briggs1:/var/crash/201809040315# dpkg -l | grep crash [...] ii crash 7.2.3+real-1~16.04.1 ppc64el kernel debugging utility, allowing gdb like syntax ii kdump-tools 1:1.6.3-2~16.04.1 ppc64el scripts and tools for automating kdump (Linux crash dumps) [...] root@ltc-briggs1:/var/crash/201809040315# root@ltc-briggs1:/var/crash/201809040315# root@ltc-briggs1:/var/crash/201809040315# uname -a Linux ltc-briggs1 4.15.0-33-generic #36~16.04.1-Ubuntu SMP Wed Aug 15 17:18:19 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux root@ltc-briggs1:/var/crash/201809040315# root@ltc-briggs1:/var/crash/201809040315# root@ltc-briggs1:/var/crash/201809040315# root@ltc-briggs1:/var/crash/201809040315# crash /usr/lib/debug/boot/vmlinux-4.15.0-33-generic dump.201809040315 crash 7.2.3 Copyright (C) 2002-2017 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "powerpc64le-unknown-linux-gnu"... KERNEL: /usr/lib/debug/boot/vmlinux-4.15.0-33-generic DUMPFILE: dump.201809040315 [PARTIAL DUMP] CPUS: 160 DATE: Tue Sep 4 03:15:08 2018 UPTIME: 00:04:15 LOAD AVERAGE: 0.90, 0.81, 0.33 TASKS: 1587 NODENAME: ltc-briggs1 RELEASE: 4.15.0-33-generic VERSION: #36~16.04.1-Ubuntu SMP Wed Aug 15 17:18:19 UTC 2018 MACHINE: ppc64le (2926 Mhz) MEMORY: 256 GB PANIC: "sysrq: SysRq : Trigger a crash" PID: 3665 COMMAND: "bash" TASK: c000003c813a0580 [THREAD_INFO: c000003c81478000] CPU: 2 STATE: TASK_RUNNING (SYSRQ) crash> bt PID: 3665 TASK: c000003c813a0580 CPU: 2 COMMAND: "bash" #0 [c000003c8147b830] crash_kexec at c0000000001e4d34 #1 [c000003c8147b870] oops_end at c000000000026318 #2 [c000003c8147b8f0] bad_page_fault at c00000000006b424 #3 [c000003c8147b960] handle_page_fault at c00000000000a650 Data Access [300] exception frame: R0: c0000000007c2d14 R1: c000003c8147bc50 R2: c0000000016baa00 R3: 0000000000000063 R4: c000001ff5c8ce18 R5: c000001ff5ca4368 R6: 9000000000009033 R7: 0000000000000012 R8: 0000000000000007 R9: 0000000000000001 R10: 0000000000000000 R11: 9000000000001003 R12: c0000000007c1b60 R13: c000000007a61600 R14: ffffffffffffffff R15: 0000000022000000 R16: 0000000010170dd0 R17: 0000010036330288 R18: 0000000010140568 R19: 00000000100c7000 R20: 0000000000000000 R21: 000000001017dd78 R22: 0000000010140400 R23: 0000000000000000 R24: 00000000101532c0 R25: 000000001017b628 R26: c0000000015b9aa8 R27: 0000000000000002 R28: 0000000000000004 R29: 0000000000000063 R30: c000000001542a9c R31: c0000000015b9e68 NIP: c0000000007c1b98 MSR: 9000000000009033 OR3: c000000000008874 CTR: c0000000007c1b60 LR: c0000000007c2d14 XER: 0000000020000000 CCR: 0000000028242222 MQ: 0000000000000001 DAR: 0000000000000000 DSISR: 0000000042000000 Syscall Result: 0000000000000000 #4 [c000003c8147bc50] sysrq_handle_crash at c0000000007c1b98 [Link Register] [c000003c8147bc50] __handle_sysrq at c0000000007c2d14 #5 [c000003c8147bc70] __handle_sysrq at c0000000007c2d14 (unreliable) #6 [c000003c8147bd10] write_sysrq_trigger at c0000000007c34d0 #7 [c000003c8147bd40] proc_reg_write at c00000000046d55c #8 [c000003c8147bd70] __vfs_write at c0000000003c1690 #9 [c000003c8147bd90] vfs_write at c0000000003c1904 #10 [c000003c8147bde0] sys_write at c0000000003c1c7c #11 [c000003c8147be30] system_call at c00000000000b284 System Call [c01] exception frame: R0: 0000000000000004 R1: 00007fffeae2d1d0 R2: 00007dea43927300 R3: 0000000000000001 R4: 0000010036311408 R5: 0000000000000002 R6: 00000000101532c2 R7: 0000010036311408 R8: 0000000000000000 R9: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 R13: 00007dea43a3af90 NIP: 00007dea4386e09c MSR: 900000000000f033 OR3: 0000000000000001 CTR: 0000000000000000 LR: 00007dea437e7dec XER: 0000000000000000 CCR: 0000000048242882 MQ: 0000000000000001 DAR: 0000010036477008 DSISR: 000000000a000000 Syscall Result: 0000000000000000 crash> -- Thanks Hari ** Tags removed: verification-needed verification-needed-xenial ** Tags added: verification-done verification-done-xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to crash in Ubuntu. https://bugs.launchpad.net/bugs/1746088 Title: [Ubuntu 16.04.4] Unable to analyze the vmcore generated by kdump on 4.13.0-26-generic kernel Status in The Ubuntu-power-systems project: Fix Committed Status in crash package in Ubuntu: Fix Released Status in crash source package in Xenial: Fix Committed Status in crash source package in Artful: Won't Fix Status in crash source package in Bionic: Fix Released Bug description: [Impact] It won't be possible to analyze dumps produced by newer kernels (hwe on xenial, for example). [Test Case] Tested that this version of crash can analyze both GA (4.4) and hwe (4.15) kernels. [Regression Potential] New crash versions may have bugs and some commands not work with older kernels. The smoke test helps a little, but more testing may be desirable. ---Problem Description--- Unable to analyze the vmcore generated by kdump on 4.13.0-26-generic kernel (Ubuntu 16.04.4) ---uname output--- Linux ltc-briggs1 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 21:40:36 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux Machine Type = 8001-22C ---Steps to Reproduce--- This bug follow up bug of https://bugzilla.linux.ibm.com/show_bug.cgi?id=163565 The steps to create dump is as follows Once you generate the kdump use crash to analyze the vmcore and we get this error ================console logs ========== root@ltc-briggs1:/var/crash/201801150227# ls dmesg.201801150227 vmcore.201801150227 .0-26-generic vmcore.2018011502271150227# crash /usr/lib/debug/boot/vmlinux-4.13. crash 7.1.4 Copyright (C) 2002-2015 Red Hat, Inc. Copyright (C) 2004, 2005, 2006, 2010 IBM Corporation Copyright (C) 1999-2006 Hewlett-Packard Co Copyright (C) 2005, 2006, 2011, 2012 Fujitsu Limited Copyright (C) 2006, 2007 VA Linux Systems Japan K.K. Copyright (C) 2005, 2011 NEC Corporation Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc. Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc. This program is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Enter "help copying" to see the conditions. This program has absolutely no warranty. Enter "help warranty" for details. GNU gdb (GDB) 7.6 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "powerpc64le-unknown-linux-gnu"... please wait... (gathering module symbol data) WARNING: cannot access vmalloc'd module memory crash: invalid structure member offset: thread_info_task FILE: task.c LINE: 598 FUNCTION: irqstacks_init() [/usr/bin/crash] error trace: 1008ade0 => 1011552c => 1017d220 => 100833e0 100833e0: (undetermined) 1017d220: OFFSET_verify+80 1011552c: task_init+5084 1008ade0: main_loop+336 == Comment from Hari Krishna Bathini == There are quite a few commits (all available upstream) that are needed for crash tool to work fine. I think the right thing to do here would be to use the latest crash tool version 7.2.0 to go with the kernel update. Also, the below commit would be needed on top of 7.2.0 crash utility: commit c8178eca9c74f81a7f803a58d339635cc152e8d9 Author: Dave Anderson <ander...@redhat.com> Date: Thu Nov 9 11:39:05 2017 -0500 Update for support of Linux 4.14 and later PPC64 kernels where the hash page table geometry accomodates a larger virtual address range. Without the patch, the virtual-to-physical translation of user space virtual addresses by "vm -p", "vtop", and "rd -u" may generate an invalid translation or otherwise fail. (hbath...@linux.vnet.ibm.com) Similar thing holds true for makedumpfile tool.. Thanks Hari To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-power-systems/+bug/1746088/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp