Re: ☠ Buildbot (GNU Toolchain): elfutils - failed test (failure) (master)
On Sat, 2022-05-28 at 11:43 +0200, Mark Wielaard wrote:
> On Sat, May 28, 2022 at 11:35:19AM +0200, Mark Wielaard wrote:
> > Retrying with ulimit -c unlimited...
> >
> > O! It failed again, now with core
> >
> > (gdb) where
> > #0 0x7f4d528be387 in __GI_raise (sig=sig@entry=6) at
> > ../nptl/sysdeps/unix/sysv/linux/raise.c:55
> > #1 0x7f4d528bfa78 in __GI_abort () at abort.c:90
> > #2 0x7f4d53dfae1f in mhd_panic_std (cls=,
> > file=,
> > line=, reason=) at daemon.c:117
> > #3 0x7f4d53dfc69e in MHD_cleanup_connections (
> > daemon=daemon@entry=0xdf38e0) at daemon.c:1826
> > #4 0x7f4d53dfdf60 in MHD_run_from_select (daemon=daemon@entry=
> > 0xdf38e0,
> > read_fd_set=read_fd_set@entry=0x7f4d4c153c10,
> > write_fd_set=write_fd_set@entry=0x7f4d4c153c90,
> > except_fd_set=except_fd_set@entry=0x7f4d4c153d10) at
> > daemon.c:2014
> > #5 0x7f4d53dfe1e9 in MHD_select (daemon=daemon@entry=0xdf38e0,
> > may_block=may_block@entry=1)
> > at daemon.c:2109
> > #6 0x7f4d53dfe3b2 in MHD_select_thread (cls=0xdf38e0) at
> > daemon.c:2632
> > #7 0x7f4d53681ea5 in start_thread (arg=0x7f4d4c154700) at
> > pthread_create.c:307
> > #8 0x7f4d52986b0d in clone () at
> > ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
> >
> > (gdb)
> > #3 0x7f4d53dfc69e in MHD_cleanup_connections (
> > daemon=daemon@entry=0xdf38e0) at daemon.c:1826
> > 1826MHD_PANIC ("close failed\n");
> > (gdb) list
> > 1821{
> > 1822#ifdef WINDOWS
> > 1823 SHUTDOWN (pos->socket_fd, SHUT_WR);
> > 1824#endif
> > 1825 if (0 != CLOSE (pos->socket_fd))
> > 1826MHD_PANIC ("close failed\n");
> > 1827}
> > 1828 if (NULL != pos->addr)
> > 1829free (pos->addr);
> > 1830 free (pos);
> >
> > Hohum. That does look like a bug in libmicrohttpd. Or do we
> > manipulate
> > the socket_fd in any way?
>
> And in case it is useful for analysis:
>
> (gdb) print *pos
> $2 = {nextE = 0x0, prevE = 0x0, next = 0x0, prev = 0x0, nextX = 0x0,
> prevX = 0x0, daemon = 0xdf38e0,
> headers_received = 0x0, headers_received_tail = 0x0, response =
> 0x0, pool = 0x7f4d242f2450,
> client_context = 0x0, method = 0x0, url = 0x0, version = 0x0,
> read_buffer = 0x7f4d242f2480 "\240\355\v$M\177", write_buffer =
> 0x0, last = 0x0, colon = 0x0,
> addr = 0x7f4d242fa490, pid = 139968370415360, read_buffer_size =
> 16384, read_buffer_offset = 0,
> write_buffer_size = 0, write_buffer_send_offset = 0,
> write_buffer_append_offset = 0,
> remaining_upload_size = 0, response_write_position = 0,
> continue_message_write_offset = 0,
> addr_len = 28, last_activity = 599293, connection_timeout = 0,
> client_aware = 0, socket_fd = 96,
> read_closed = 1, thread_joined = 0, in_idle = 0, epoll_state =
> MHD_EPOLL_STATE_UNREADY,
> state = MHD_CONNECTION_CLOSED, event_loop_info =
> MHD_EVENT_LOOP_INFO_CLEANUP, responseCode = 0,
> response_unready = 0, have_chunked_upload = 0, current_chunk_size =
> 0, current_chunk_offset = 0,
> read_handler = 0x7f4d53df7680 ,
> write_handler = 0x7f4d53df78d0 ,
> idle_handler = 0x7f4d53df83a0 ,
> recv_cls = 0x7f4d53dfa030 , send_cls =
> 0x7f4d53df9ec0 ,
> tls_session = 0x0, protocol = 0, cipher = 0, tls_read_ready = 0,
> suspended = 0, resuming = 0}
>
> $ rpm -q libmicrohttpd
> libmicrohttpd-0.9.33-2.el7.x86_64
I tried to debug this a bit. But the code is really different from any
more recent version of libmicrohttpd. Somewhere libmicrohttpd looses
track of the state of the file descriptor and then it just aborts in
cleanup. I couldn't find a way to work around it. The version is so old
that it doesn't actually use any of the modern flags (MHD_USE_EPOLL,
MHD_USE_INTERNAL_POLLING_THREAD and MHD_USE_ITC are all not available).
Even reducing the number of fetches in run-debuginfod-federation-
metrics.sh doesn't prevent it from crashing.
I gave up and just wrote this ugly version based patch to not run the
testcase on such old libmicrohttpd.
Cheers,
Mark
From 8cbadd86c147eeaec4344ac65ad00ea96ae1451a Mon Sep 17 00:00:00 2001
From: Mark Wielaard
Date: Thu, 2 Jun 2022 17:36:39 +0200
Subject: [PATCH] tests: Disable run-debuginfod-federation-metrics.sh for old
libmicrohttpd
On really old libmicrohttpd the run-debuginfod-federation-metrics.sh
test will crash debuginfod after too many file descriptors have been
used. libmicrohttpd looses track of the state and aborts instead of
producing an error. Just disable the testcase on these very old versions.
Signed-off-by: Mark Wielaard
---
ChangeLog | 5 +
configure.ac | 2 ++
tests/ChangeLog | 5 +
tests/Makefile.am | 6 +-
4 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/ChangeLog b/ChangeLog
index f1a14b5c..c5e43e8c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,8 @@
+2022-06-02 Mark Wielaard
+
+ * configure.ac (OLD_LIB
☺ Buildbot (GNU Toolchain): elfutils - build successful (master)
A restored build has been detected on builder elfutils-centos-x86_64 while building elfutils. Full details are available at: https://builder.sourceware.org/buildbot/#builders/39/builds/34 Build state: build successful Revision: 8cbadd86c147eeaec4344ac65ad00ea96ae1451a Worker: centos-x86_64 Build Reason: (unknown) Blamelist: Mark Wielaard Steps: - 0: worker_preparation ( success ) - 1: set package name ( success ) - 2: git checkout ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/2/logs/stdio - 3: autoreconf ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/3/logs/stdio - 4: configure ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/4/logs/stdio - 5: get version ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/5/logs/stdio - property changes: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/5/logs/property_changes - 6: make ( warnings ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/6/logs/stdio - warnings (2): https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/6/logs/warnings__2_ - 7: make check ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/7/logs/stdio - test-suite.log: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/7/logs/test-suite_log - 8: prep ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/8/logs/stdio - 9: fetch ['tests/*.trs', 'tests/*.log'] ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/9/logs/stdio - 10: fetch ['elfutils-*/_build/sub/tests/*.trs', 'elfut ( skipped ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/10/logs/stdio - 11: fetch ['config.log'] ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/11/logs/stdio - 12: fetch ['elfutils-*/_build/sub/config.log'] ( skipped ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/12/logs/stdio - 13: pass .bunsen.source.gitname ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/13/logs/stdio - 14: pass .bunsen.source.gitrepo ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/14/logs/stdio - 15: upload to bunsen ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/15/logs/stdio - 16: clean up ( success ) Logs: - stdio: https://builder.sourceware.org/buildbot/#builders/39/builds/34/steps/16/logs/stdio
Re: debuginfod & Debian source packages
Hi - > I'm the maintainer of debuginfod.debian.net (currently offline due to a > hardware issue :-/). O no. (And also, please consider upgrading your elfutils version, as later ones have less pessimal behavior with long grooming ops.) > The service provides only debuginfo for now, but I > would like to start indexing the source code of each package as well. Neat! > [...] > I'm now thinking about an alternative solution to this problem. Debian > source packages already contain everything needed to be indexed and > served to debuginfo consumers; it has the full source code + all the > downstream patches. It's represented by a .dsc file that can be fed to > dget(1) which will download the source tarball and apply all the patches > automatically. > > Do you think we can teach debuginfod to consume these files and do the > right thing when indexing the source code of each package? [...] Interesting idea, but I'd throw it back at you thusly: does debuginfod need to this itself on the fly? Other than perhaps a time/storage tradeoff, is there some reason an auxiliary program couldn't do this in the background? Take designated .dsc's, download, apply patches, and assemble the patched sources into, well, any old random tarball format? If someone(tm) were to write such a script, we could ship it alongside debuginfod if desired. As long as the archive file name was a close match to the binary deb file names, and as long as the constituent files were named with the same paths as mentioned in the binary dwarf, debuginfod would gladly take the result and serve from it. - FChE
Re: debuginfod & Debian source packages
On Thursday, June 02 2022, Frank Ch. Eigler wrote: > Hi - Hi Frank, >> I'm the maintainer of debuginfod.debian.net (currently offline due to a >> hardware issue :-/). > > O no. (And also, please consider upgrading your elfutils version, as > later ones have less pessimal behavior with long grooming ops.) I have to backport the new version to Debian bullseye, but yeah, this is on my TODO list. >> [...] >> I'm now thinking about an alternative solution to this problem. Debian >> source packages already contain everything needed to be indexed and >> served to debuginfo consumers; it has the full source code + all the >> downstream patches. It's represented by a .dsc file that can be fed to >> dget(1) which will download the source tarball and apply all the patches >> automatically. >> >> Do you think we can teach debuginfod to consume these files and do the >> right thing when indexing the source code of each package? [...] > > Interesting idea, but I'd throw it back at you thusly: does debuginfod > need to this itself on the fly? Other than perhaps a time/storage > tradeoff, is there some reason an auxiliary program couldn't do this > in the background? Take designated .dsc's, download, apply patches, > and assemble the patched sources into, well, any old random tarball > format? If someone(tm) were to write such a script, we could ship it > alongside debuginfod if desired. Sure, I believe an external script/program could do things behind the scenes without problem. > As long as the archive file name was a close match to the binary deb > file names, and as long as the constituent files were named with the > same paths as mentioned in the binary dwarf, debuginfod would gladly > take the result and serve from it. OK, this was going to be my next question. Out of curiosity, how would debuginfod invoke this external program? Thanks, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible https://sergiodj.net/
Re: debuginfod & Debian source packages
Hi - > [...] > OK, this was going to be my next question. Out of curiosity, how would > debuginfod invoke this external program? That's the best part. It doesn't need to. :-) Whatever program you're using to freshen the repository of .deb's that your debuginfod scrapes, could also run this program for any new .dsc's. i.e., keep the problem from debuginfod entirely. - FChE
Re: debuginfod & Debian source packages
On Thursday, June 02 2022, Frank Ch. Eigler wrote: > Hi - > >> [...] >> OK, this was going to be my next question. Out of curiosity, how would >> debuginfod invoke this external program? > > That's the best part. It doesn't need to. :-) Whatever program you're > using to freshen the repository of .deb's that your debuginfod > scrapes, could also run this program for any new .dsc's. i.e., keep > the problem from debuginfod entirely. Ah, OK. Yeah, I guess it makes sense to do it this way. I will do some hacking this coming weekend and see what I come up with, and then I will let you know. Hopefully this will turn out to be easier than I thought. Thanks, -- Sergio GPG key ID: 237A 54B1 0287 28BF 00EF 31F4 D0EB 7628 65FC 5E36 Please send encrypted e-mail if possible https://sergiodj.net/
