Re: [PATCH 4/4] strip: Add --reloc-debug-sections-only option.
On Tue, Nov 06, 2018 at 03:21:24PM +0100, Mark Wielaard wrote:
> The buildbot didn't sent any failure emails (I am still looking into
> why), but this new cmp test failed on debian-armhf, fedora-ppc64 and
> fedora-ppc64le:
> https://builder.wildebeest.org/buildbot/#/builders/15/builds/81
> https://builder.wildebeest.org/buildbot/#/builders/12/builds/242
> https://builder.wildebeest.org/buildbot/#/builders/11/builds/244
>
> It succeeds on everything else. Still investigating.
The buildbot did sent email. I was too fast.
The issue was kind of subtle, but is easily fixable.
I committed the attached patch with some extra testcases to
(hopefully) get the build green again on all arches.
Cheers,
Mark
>From c6e3290a9722df58191c1ac181ce587f406d7b7a Mon Sep 17 00:00:00 2001
From: Mark Wielaard
Date: Fri, 9 Nov 2018 08:18:22 +
Subject: [PATCH] libelf: Explicitly update section data after (de)compression.
We need to explictly trigger a section data reload after updating the
ELF section rawdata to make sure it gets written out to disk on an
elf_update. Doing this showed one bug/inefficiently when the underlying
file has a different endianness. In that case for debug sections we
would convert by allocating a new buffer and just copying over the
raw data into a new buffer. This is not really necessary and would
hide any relocations done on the rawdata by libdwfl.
Added a couple of new ppc64 big endian testfiles that show the issue.
Signed-off-by: Mark Wielaard
---
libelf/ChangeLog | 7 ++
libelf/elf_compress.c | 6 ++
libelf/elf_getdata.c | 3 +-
tests/ChangeLog| 10 +++
tests/Makefile.am | 5 +-
tests/run-readelf-zdebug-rel.sh| 106 +
tests/run-strip-reloc.sh | 4 +
tests/testfile-debug-rel-ppc64-g.o.bz2 | Bin 0 -> 1400 bytes
tests/testfile-debug-rel-ppc64-z.o.bz2 | Bin 0 -> 1420 bytes
tests/testfile-debug-rel-ppc64.o.bz2 | Bin 0 -> 1103 bytes
10 files changed, 139 insertions(+), 2 deletions(-)
create mode 100644 tests/testfile-debug-rel-ppc64-g.o.bz2
create mode 100644 tests/testfile-debug-rel-ppc64-z.o.bz2
create mode 100644 tests/testfile-debug-rel-ppc64.o.bz2
diff --git a/libelf/ChangeLog b/libelf/ChangeLog
index af5650360..53da9a65f 100644
--- a/libelf/ChangeLog
+++ b/libelf/ChangeLog
@@ -1,3 +1,10 @@
+2018-11-09 Mark Wielaard
+
+ * elf_compress.c (__libelf_reset_rawdata): Make rawdata change
+ explicit by calling __libelf_set_data_list.
+ * elf_getdata.c (convert_data): Don't convert if type is ELF_T_BYTE
+ even if endianness is different.
+
2018-10-18 Mark Wielaard
* libelf.h (Elf_Type): Add ELF_T_NHDR8.
diff --git a/libelf/elf_compress.c b/libelf/elf_compress.c
index fd412e8a6..d96245df2 100644
--- a/libelf/elf_compress.c
+++ b/libelf/elf_compress.c
@@ -326,6 +326,12 @@ __libelf_reset_rawdata (Elf_Scn *scn, void *buf, size_t size, size_t align,
scn->rawdata_base = buf;
scn->flags |= ELF_F_MALLOCED;
+
+ /* Pretend we (tried to) read the data from the file and setup the
+ data (might have to convert the Chdr to native format). */
+ scn->data_read = 1;
+ scn->flags |= ELF_F_FILEDATA;
+ __libelf_set_data_list_rdlock (scn, 1);
}
int
diff --git a/libelf/elf_getdata.c b/libelf/elf_getdata.c
index 4f80aaf2d..2043bba2d 100644
--- a/libelf/elf_getdata.c
+++ b/libelf/elf_getdata.c
@@ -146,7 +146,8 @@ convert_data (Elf_Scn *scn, int version __attribute__ ((unused)), int eclass,
{
const size_t align = __libelf_type_align (eclass, type);
- if (data == MY_ELFDATA)
+ /* Do we need to convert the data and/or adjust for alignment? */
+ if (data == MY_ELFDATA || type == ELF_T_BYTE)
{
if (size_t) (char *) scn->rawdata_base)) & (align - 1)) == 0)
/* No need to copy, we can use the raw data. */
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 23e911336..7ce39808d 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,3 +1,13 @@
+2018-11-09 Mark Wielaard
+
+ * testfile-debug-rel-ppc64-g.o.bz2: New test file.
+ * testfile-debug-rel-ppc64-z.o.bz2: Likewise.
+ * testfile-debug-rel-ppc64.o.bz2: Likewise.
+ * Makefile.am (EXTRA_DIST): Add testfile-debug-rel-ppc64-g.o.bz2,
+ testfile-debug-rel-ppc64-z.o.bz2 and testfile-debug-rel-ppc64.o.bz2.
+ * run-strip-reloc.sh: Also test on testfile-debug-rel-ppc64.o.
+ * run-readelf-zdebug-rel.sh: Also test on testfile-debug-rel-ppc64*.o.
+
2018-10-26 Mark Wielaard
* run-strip-reloc.sh: Add a test for --reloc-debug-sections-only.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a2a381ac8..d3ac345df 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -412,7 +412,10 @@ EXTRA_DIST = run-arextract.sh run-arsymtest.sh run-ar.sh \
testfile-riscv64.bz2 testfile-riscv64-s.bz2 \
testfile-riscv64-core.bz2 \
run-copyadd-sections.sh run-copymany-sections.sh \
- run-typeiter-many.sh run-strip-test-many.sh
+ run-ty
[PATCH] strip: Also handle gnu compressed debug sections with --reloc-debug-sections
Check whether a section was gnu compressed and decompress it first
before trying to resolve relocations. Recompress it afterwards.
This found a bug in elf_compress_gnu which would use the "raw" file
contents even if the user had just created the section (copying over
the section from the original input file).
Add compressed ET_REL tests to run-strip-reloc.sh testcase.
Signed-off-by: Mark Wielaard
---
libelf/ChangeLog | 4
libelf/elf_compress_gnu.c | 7 ---
src/ChangeLog | 5 +
src/strip.c | 29 +++--
tests/ChangeLog | 5 +
tests/run-strip-reloc.sh | 6 ++
6 files changed, 47 insertions(+), 9 deletions(-)
diff --git a/libelf/ChangeLog b/libelf/ChangeLog
index 53da9a6..30760bf 100644
--- a/libelf/ChangeLog
+++ b/libelf/ChangeLog
@@ -1,5 +1,9 @@
2018-11-09 Mark Wielaard
+ * elf_compress_gnu.c (elf_compress_gnu): Use elf_getdata.
+
+2018-11-09 Mark Wielaard
+
* elf_compress.c (__libelf_reset_rawdata): Make rawdata change
explicit by calling __libelf_set_data_list.
* elf_getdata.c (convert_data): Don't convert if type is ELF_T_BYTE
diff --git a/libelf/elf_compress_gnu.c b/libelf/elf_compress_gnu.c
index 198dc7d..1ecd6a0 100644
--- a/libelf/elf_compress_gnu.c
+++ b/libelf/elf_compress_gnu.c
@@ -144,9 +144,10 @@ elf_compress_gnu (Elf_Scn *scn, int inflate, unsigned int
flags)
else if (inflate == 0)
{
/* In theory the user could have constucted a compressed section
-by hand. But we always just take the rawdata directly and
-decompress that. */
- Elf_Data *data = elf_rawdata (scn, NULL);
+by hand. And in practice they do. For example when copying
+a section from one file to another using elf_newdata. So we
+have to use elf_getdata (not elf_rawdata). */
+ Elf_Data *data = elf_getdata (scn, NULL);
if (data == NULL)
return -1;
diff --git a/src/ChangeLog b/src/ChangeLog
index 79e6872..e8b7ad8 100644
--- a/src/ChangeLog
+++ b/src/ChangeLog
@@ -1,3 +1,8 @@
+2018-11-09 Mark Wielaard
+
+ * strip.c (remove_debug_relocations): Check if section is gnu
+ compressed and decompress and recompress it.
+
2018-10-26 Mark Wielaard
* strip.c (OPT_RELOC_DEBUG_ONLY): New define.
diff --git a/src/strip.c b/src/strip.c
index e953c4d..1518073 100644
--- a/src/strip.c
+++ b/src/strip.c
@@ -485,12 +485,22 @@ remove_debug_relocations (Ebl *ebl, Elf *elf, GElf_Ehdr
*ehdr,
(and recompress if necessary at the end). */
GElf_Chdr tchdr;
int tcompress_type = 0;
- if (gelf_getchdr (tscn, &tchdr) != NULL)
+ bool is_gnu_compressed = false;
+ if (strncmp (tname, ".zdebug", strlen ("zdebug")) == 0)
{
- tcompress_type = tchdr.ch_type;
- if (elf_compress (tscn, 0, 0) != 1)
+ is_gnu_compressed = true;
+ if (elf_compress_gnu (tscn, 0, 0) != 1)
INTERNAL_ERROR (fname);
}
+ else
+ {
+ if (gelf_getchdr (tscn, &tchdr) != NULL)
+ {
+ tcompress_type = tchdr.ch_type;
+ if (elf_compress (tscn, 0, 0) != 1)
+ INTERNAL_ERROR (fname);
+ }
+ }
Elf_Data *tdata = elf_getdata (tscn, NULL);
if (tdata == NULL || tdata->d_buf == NULL
@@ -686,9 +696,16 @@ remove_debug_relocations (Ebl *ebl, Elf *elf, GElf_Ehdr
*ehdr,
shdr->sh_size = reldata->d_size = nrels * shdr->sh_entsize;
gelf_update_shdr (scn, shdr);
- if (tcompress_type != 0)
- if (elf_compress (tscn, tcompress_type, ELF_CHF_FORCE) != 1)
- INTERNAL_ERROR (fname);
+ if (is_gnu_compressed)
+ {
+ if (elf_compress_gnu (tscn, 1, ELF_CHF_FORCE) != 1)
+ INTERNAL_ERROR (fname);
+ }
+ else if (tcompress_type != 0)
+ {
+ if (elf_compress (tscn, tcompress_type, ELF_CHF_FORCE) != 1)
+ INTERNAL_ERROR (fname);
+ }
}
}
}
diff --git a/tests/ChangeLog b/tests/ChangeLog
index 7ce3980..7472bc2 100644
--- a/tests/ChangeLog
+++ b/tests/ChangeLog
@@ -1,5 +1,10 @@
2018-11-09 Mark Wielaard
+ * run-strip-reloc.sh: Also test testfile-debug-rel-ppc64-z.o
+ testfile-debug-rel-ppc64-g.o.
+
+2018-11-09 Mark Wielaard
+
* testfile-debug-rel-ppc64-g.o.bz2: New test file.
* testfile-debug-rel-ppc64-z.o.bz2: Likewise.
* testfile-debug-rel-ppc64.o.bz2: Likewise.
diff --git a/tests/run-strip-reloc.sh b/tests/run-strip-reloc.sh
index 6f299ab..0c6b1c2 100755
--- a/tests/run-strip-reloc.sh
+++ b/tests/run-strip-reloc.sh
@@ -139,4 +139,10 @@ runtest strip-compressed.o 1
testfiles testfile-debug-rel-ppc64.o
runtest testfile-debug-rel-ppc64.o 1
+testfiles testfile-debug-rel-ppc64-z.o
+runtest testfile-debug-re
Re: [PATCH] Also find CFI in sections of type SHT_X86_64_UNWIND
On Wed, Nov 07, 2018 at 10:02:54AM +0100, Milian Wolff wrote: > On Dienstag, 6. November 2018 12:06:57 CET Mark Wielaard wrote: > > It seems to only happen with a specific combination of gcc and the gold > > linker, I could only generate the SHT_X86_64_UNWIND sections only on > > fedora 29 with gcc 8.2.1 and gold version 2.31.1-13.fc29 (1.16). > > At least on my system, that doesn't seem to be enough. Both my desktop and my > laptop are running on ArchLinux with the same versions of GCC and Gold, yet > one shows this behavior while the other one isn't... Strange indeed. My system doesn't have ccache, it does show the issue. > Both patches work for me: > > Tested-by: Milian Wolff Great. I pushed both to master now. Thanks, Mark
Re: [PATCH] libdwelf: New function dwelf_elf_begin.
On Sun, Nov 04, 2018 at 05:37:42PM +0100, Mark Wielaard wrote: > On Mon, 2018-10-22 at 01:47 +0200, Mark Wielaard wrote: > > It currently wraps __libdw_open_file which makes error handling > > slight tricky and needs duplicating the file handle. > > Which introduced some confusion that could cause the file descriptor to > leak. The attached updated patch adds a variant to the > __libdw_open_file function that doesn't try to play tricks with the > given file descriptor which is easier to wrap for dwelf_elf_begin. > From 940515f0ba9641b9ac38d721aaac87fd898bb1fd Mon Sep 17 00:00:00 2001 > From: Mark Wielaard > Date: Sun, 21 Oct 2018 23:41:32 +0200 > Subject: [PATCH] libdwelf: New function dwelf_elf_begin. > > This introduces a new function dwelf_elf_begin which creates a (read-only) > ELF handle from a possibly compressed file handle or a file that start > with a linux kernel header. This can be used in eu-readelf to (re)open a > (pure) ELF. > > eu-readelf uses libdwfl to relocate addresses in the original file in > case it is ET_REL. But to show the "raw" data it might need to (re)open > the file. Which could fail if the file was compressed. And produced an > obscure error message: "cannot create EBL handle". > > This rewrites __libdw_open_file a little so that the given file handle > will never be closed (whether on success or failure) and introduces a > new internal function __libdw_open_elf that dwelf_elf_begin wraps. I pushed this variant to master now. Cheers, Mark
Re: [PATCH] libcpu: Recognize bpf jump variants BPF_JLT, BPF_JLE, BPF_JSLT and BPF_JSLE
On Sun, Nov 04, 2018 at 09:40:32PM +0100, Mark Wielaard wrote:
> Linux kernel 4.13 introduced 4 more jump class variants.
>
> commit 92b31a9af73b3a3fc801899335d6c47966351830
> Author: Daniel Borkmann
> Date: Thu Aug 10 01:39:55 2017 +0200
>
> bpf: add BPF_J{LT,LE,SLT,SLE} instructions
>
> For conditional jumping on unsigned and signed < and <= between a register
> and another register or immediate.
>
> Add these new constants to bpf.h, recognize them in bpf_disasm and update
> the testfile-bpf-dis1.expect file.
Pushed to master.
