[Bug libdw/22452] New: Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 Bug ID: 22452 Summary: Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module Product: elfutils Version: unspecified Status: UNCONFIRMED Severity: normal Priority: P2 Component: libdw Assignee: unassigned at sourceware dot org Reporter: brueckner at linux dot vnet.ibm.com CC: elfutils-devel at sourceware dot org Target Milestone: --- Created attachment 10612 --> https://sourceware.org/bugzilla/attachment.cgi?id=10612&action=edit ko-func-call-frame-cfa.c Problem --- The perf probe -m -V fails when looking up variables for a function defined in an s390 kernel module. I have attached a simple test program that performs these steps which results in problems to receive an CFI frame. Test Case and reproduction steps 1. Iterate through the DIE to spot a particular function 2. Determine frame base attribute for that function 3. Obtain location expressions for the frame base and entry pc of the function 4. The returned location expression is a DW_OP_call_frame_cfa 5. Obtain CFI from either eh_frame or debug_frame, which fails. 6. Obtain ops from dwarf_frame_cfa() The step 5 always fails for both, .eh_frame and debug_frame. When using dwarf_cfi_addrframe() on the .eh_frame, the resulting dwarf error message is "no matching address range". The attachment "ko-func-call-frame-cfa.c" contains this test case. Compile and link against ldw -lelf and issue with: ./ko-func-call-frame-cfa Note that this affects kernel modules built for s390. Running the test program against the s390 vmlinux image works fine, e.g., for the setup_arch function. As I cannot attach binary code, you can obtain an s390 kernel module, for example, from the kernel debuginfo packages of Fedora. For example: 1. https://download-ib01.fedoraproject.org/pub/fedora-secondary/releases/27/Server/s390x/debug/tree/Packages/k/kernel-debuginfo-4.13.9-300.fc27.s390x.rpm 2. rpm2cpio < kernel-debuginfo-4.13.9-300.fc27.s390x.rpm |cpio -id 3. Try to find details about the "xts_paes_crypt" function in paes_s390.ko.debug For example: ko-func-call-frame-cfa xts_paes_crypt kernel/arch/s390/crypto/paes_s390.ko.debug I: Looking up xts_paes_crypt@kernel/arch/s390/crypto/paes_s390.ko.debug DIE: name=xts_paes_crypt tag=subprogram offset=58750 (0xe57e) low pc: 10c18 high pc: 10dd6 entry pc: 10c18 I: Frame Base: Number of locations: 1 (len=1) I: Frame Base: Location operation 0: call_frame_cfa E: No CFI data available -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #2 from H. Brueckner --- (In reply to Mark Wielaard from comment #1) > I haven't investigated fully yet. > But I did take a quick look at the paes_s390.ko.debug file. > A few quick comments as early feedback. > > Note that it doesn't contain .debug_frame but only a .eh_frame section. > Other architectures force .debug_frame for kernel modules to store the CFI. > > The .eh_frame is a loaded section, which means it won't go into the .debug > file. So you would need to load the CFI from the main ELF .ko. Initially, I have tried on my own kernel build (including debug_info) and ran the test program against the .ko file. (Because I am not permitted to share binary code and, please, do not ask why, it is difficult to provide you an alternative. If you have access to a s390 or use a cross-compiler it would be fine but that typically requires some effort on your side. In any case, if you require some output in text form, please let me know. > One tricky issue here is the fact that kernel modules are ET_REL files > that still need relocations applied (you can see there is also a > .rela.eh_frame section). libdwfl can do simple relocations, but might not > know about all relocation types used in an .eh_frame section (I haven't > checked yet). Ok. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #3 from H. Brueckner --- (In reply to Mark Wielaard from comment #1) > One tricky issue here is the fact that kernel modules are ET_REL files > that still need relocations applied (you can see there is also a > .rela.eh_frame section). libdwfl can do simple relocations, but might not > know about all relocation types used in an .eh_frame section (I haven't > checked yet). At least, I checked the reloc types for s390 (backends/s390_reloc.def) and the list is complete wrt kernel modules. Not sure if there is further work to be done for non-simple relocations. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #6 from H. Brueckner --- Created attachment 10613 --> https://sourceware.org/bugzilla/attachment.cgi?id=10613&action=edit paes_s390_ko-relocs-eu-readelf.txt using eu-readelf -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #7 from H. Brueckner --- Created attachment 10614 --> https://sourceware.org/bugzilla/attachment.cgi?id=10614&action=edit paes_s390_ko-relocs-readelf.txt and what binutils readelfs reports for comparision -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #8 from H. Brueckner --- (In reply to Mark Wielaard from comment #5) > (In reply to Mark Wielaard from comment #4) > > So, if you could provide the output of eu-readelf --relocs paes_s390.ko (on > > your combined paes_s390.ko containing both code and debug) that would be > > helpful. > > eu-readelf --debug-dump=frames --relocs paes_s390.ko > > would be even better. Just attached the output; also the output of the binutils readelf command for comparision. I guess that there are some more issues because lots of invalid relocations are listed. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #10 from H. Brueckner --- (In reply to Mark Wielaard from comment #9) > (In reply to H. Brueckner from comment #8) > > (In reply to Mark Wielaard from comment #5) > > > (In reply to Mark Wielaard from comment #4) > > > > So, if you could provide the output of eu-readelf --relocs paes_s390.ko > > > > (on > > > > your combined paes_s390.ko containing both code and debug) that would be > > > > helpful. > > > > > > eu-readelf --debug-dump=frames --relocs paes_s390.ko > > > > > > > Just attached the output; also the output of the binutils readelf command > > for comparision. I guess that there are some more issues because lots of > > invalid relocations are listed. > > Is that eu-readelf from a distro install? Then there is something wrong with > that install. If it is a build from source then make sure LD_LIBRARY_PATH > contains the backends directory that has libebl_s390.so. Otherwise > eu-readelf won't know about machine specific ELF/DWARF mappings (like the > relocation types/names). It was a local built and install, the LD_LIBRARY_PATH was not set properly. However, setting is correctly, the relocation are displayed like with readelf. Will update the attachment soon. > > But the output does confirm what I feared. The R_390_PC32 .rela.eh_frame > relocations are not "simple". They are load based. So now it comes to that point I have feared, as it looks like the libdw just performs "simple" relocations. Your comment and the ebl_reloc_simple_type() call in libdwfl/relocate.c proofs that. Anyhow, do you plan to extend the relocation code to support more complex relocations tool? > It might not be too hard to add support for those. What needs to be done / is necessary for that? > But if there is any > possibility to generate .debug_frame based CFI for the s390x kernel modules > that would IMHO a much better/simpler solution because it makes things more > in line with other arches. It should also make the kernel modules smaller. > Does anything actually use the loaded .eh_frame CFI on s390x? I will check the module usage on .eh_frame and will have a try on including the CFI in the .debug_frame. Thanks. -- You are receiving this mail because: You are on the CC list for the bug.
[Bug libdw/22452] Failed to obtain CFI data for a OP_call_frame_CFA when looking for func entrypoc in an s390 kernel module
https://sourceware.org/bugzilla/show_bug.cgi?id=22452 --- Comment #14 from H. Brueckner --- Hi Mark, Andreas, (In reply to Andreas Krebbel from comment #11) > x86 enables generation of .debug_frame by using: > > KBUILD_CFLAGS += -fno-asynchronous-unwind-tables > > in the kernel Makefile. This makes GCC to add ".cfi_sections .debug_frame" > to the assembler output which in turn triggers gas to generate the section. > > I don't see a reason why we couldn't do the same for s390. The .eh_frame > section aren't used in the kernel. Thanks for this direction: Emitting the CFI data in the .debug_frame work nicely. With that change, perf probe and my test program works. For reference: ./ko-func-call-frame-cfa xts_paes_crypt ~/git/linux/arch/s390/crypto/paes_s390.ko I: Looking up xts_paes_crypt@/root/git/linux/arch/s390/crypto/paes_s390.ko DIE: name=xts_paes_crypt tag=subprogram offset=53957 (0xd2c5) low pc: 10ca8 high pc: 10e78 entry pc: 10ca8 I: Frame Base: Number of locations: 1 (len=1) I: Frame Base: Location operation 0: call_frame_cfa E: Could not obtain CFI (eh): no DWARF information I: CFA operation: 0: bregx clearly shows a successful call to dwarf_frame_cfa() obtaining the CFI from the .debug_frame (and not longe from the .eh_frame). Having explicit ebl hooks would still be nice but I guess will not be the top-priority item in the TODO list. I pursue to integrate my change for s390 in the upstream kernel. Thanks! -- You are receiving this mail because: You are on the CC list for the bug.
