Dear Maintainer, hello Mehturt, I guess the missing debug information is contained in libcurl3-dbg [1].
I tried to find out to which line this address in Mehturt's backtrace points to and came to this location: (gdb) bt #0 0x00007ffff77cd4d7 in Curl_close (data=0x55555579f6c0) at url.c:399 #1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538 #2 main (argv=<optimized out>) at ./methods/https.cc:546 https://sources.debian.org/src/curl/7.52.1-5+deb9u10/lib/url.c/#L399 https://sources.debian.org/src/apt/1.4.10/methods/https.cc/#L538 In my opinion the instruction "mov 0x60(%rbx),%rdi" is executed, for Mehturt, while $rbx contains no valid pointer. The crash did not show up in my test VM. Attached an example debug session to debug to this location. @Mehturt: maybe you can still install libcurl3-dbg and inspect the core again. (Maybe adding a "display/i $pc" and "print/x $rbx", before any "bt full") Maybe related, upstream has added a null pointer check to Curl_close. https://sources.debian.org/src/curl/7.68.0-1/lib/url.c/#L325 Kind regards, Bernhard [1] https://snapshot.debian.org/package/curl/7.52.1-5%2Bdeb9u10/#libcurl3-gnutls_7.52.1-5:2b:deb9u10
# Stretch/oldstable amd64 qemu VM 2020-05-23 #sources.list deb http://192.168.178.25:9999/debian-9-stretch-debug.mirrors.debian.org/ stretch-debug main deb http://192.168.178.25:9999/debian-9-stretch-debug.mirrors.debian.org/ stretch-proposed-updates-debug main contrib non-free #approx.conf debian-9-stretch-deb.debian.org http://deb.debian.org/debian/ debian-9-stretch-security.debian.org http://security.debian.org/ debian-9-stretch-debug.mirrors.debian.org http://debug.mirrors.debian.org/debian-debug/ # or https://wiki.debian.org/AutomaticDebugPackages apt update apt dist-upgrade apt install systemd-coredump # https://github.com/matrix-org/synapse/blob/master/INSTALL.md apt install -y lsb-release wget apt-transport-https wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/matrix-org.list apt update apt install matrix-synapse-py3 # no crash root@debian:~# dpkg -S /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 libcurl3-gnutls:amd64: /usr/lib/x86_64-linux-gnu/libcurl-gnutls.so.4 root@debian:~# dpkg -l | grep libcurl3-gnutls ii libcurl3-gnutls:amd64 7.52.1-5+deb9u10 amd64 easy-to-use client-side URL transfer library (GnuTLS flavour) apt install gdb apt-transport-https-dbgsym libcurl3-dbg rm matrix-synapse-py3_1.13.0+stretch1_amd64.deb gdb -q set width 0 set pagination off file /usr/bin/apt set detach-on-fork off set follow-fork-mode child set follow-exec-mode new b HttpsMethod::~HttpsMethod y run download matrix-synapse-py3 info inferiors inferior 1 cont info inferiors inferior 1 set detach-on-fork on cont info inferiors bt cont stepi stepi benutzer@debian:~$ rm matrix-synapse-py3_1.13.0+stretch1_amd64.deb ^C benutzer@debian:~$ gdb -q (gdb) set width 0 (gdb) set pagination off (gdb) file /usr/bin/apt Reading symbols from /usr/bin/apt...(no debugging symbols found)...done. (gdb) set detach-on-fork off (gdb) set follow-fork-mode child (gdb) set follow-exec-mode new (gdb) b HttpsMethod::~HttpsMethod Function "HttpsMethod::~HttpsMethod" not defined. Make breakpoint pending on future shared library load? (y or [n]) y Breakpoint 1 (HttpsMethod::~HttpsMethod) pending. (gdb) run download matrix-synapse-py3 Starting program: /usr/bin/apt download matrix-synapse-py3 [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New process 3044] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Reading symbols from /usr/lib/debug/.build-id/77/5143e680ff0cd4cd51cce1ce8ca216e635a1d6.debug...done. Reading symbols from /usr/lib/debug/.build-id/db/2caeeec37482a98ab1416d0a9afe2944930de9.debug...done. Reading symbols from /usr/lib/debug/.build-id/ea/d5fd817712e63c1212d1ee7d7ee1b9c29f93a7.debug...done. Reading symbols from /usr/lib/debug/.build-id/16/d609487bcc4acbac29a4eaa2dda0d2f56211ec.debug...done. Reading symbols from /usr/lib/debug/.build-id/4e/49714c557ce0472c798f39365ca10f9c0e1933.debug...done. Reading symbols from /usr/lib/debug/.build-id/60/6df9c355103e82140d513bc7a25a635591c153.debug...done. process 3044 is executing new program: /usr/bin/dpkg [New process 3044] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [Inferior 3 (process 3044) exited normally] (gdb) info inferiors Num Description Executable 1 process 3040 /usr/bin/apt * 3 <null> /usr/bin/dpkg (gdb) inferior 1 [Switching to inferior 1 [process 3040] (/usr/bin/apt)] [Switching to thread 1.1 (Thread 0x7ffff7febb80 (LWP 3040))] #0 0x00007ffff6f4a38b in __libc_fork () at ../sysdeps/nptl/fork.c:135 135 ../sysdeps/nptl/fork.c: Datei oder Verzeichnis nicht gefunden. (gdb) cont Continuing. [New process 3045] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Reading symbols from /usr/lib/debug/.build-id/77/5143e680ff0cd4cd51cce1ce8ca216e635a1d6.debug...done. Reading symbols from /usr/lib/debug/.build-id/db/2caeeec37482a98ab1416d0a9afe2944930de9.debug...done. Reading symbols from /usr/lib/debug/.build-id/ea/d5fd817712e63c1212d1ee7d7ee1b9c29f93a7.debug...done. Reading symbols from /usr/lib/debug/.build-id/16/d609487bcc4acbac29a4eaa2dda0d2f56211ec.debug...done. Reading symbols from /usr/lib/debug/.build-id/4e/49714c557ce0472c798f39365ca10f9c0e1933.debug...done. Reading symbols from /usr/lib/debug/.build-id/60/6df9c355103e82140d513bc7a25a635591c153.debug...done. process 3045 is executing new program: /usr/bin/dpkg [New process 3045] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [Inferior 5 (process 3045) exited normally] (gdb) info inferiors Num Description Executable 1 process 3040 /usr/bin/apt 3 <null> /usr/bin/dpkg * 5 <null> /usr/bin/dpkg (gdb) inferior 1 [Switching to inferior 1 [process 3040] (/usr/bin/apt)] [Switching to thread 1.1 (Thread 0x7ffff7febb80 (LWP 3040))] #0 0x00007ffff6f4a38b in __libc_fork () at ../sysdeps/nptl/fork.c:135 135 in ../sysdeps/nptl/fork.c (gdb) set detach-on-fork on (gdb) cont Continuing. [New process 3046] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". process 3046 is executing new program: /usr/lib/apt/methods/https [New process 3046] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Thread 7.1 "https" received signal SIGINT, Interrupt. [Switching to Thread 0x7ffff7fe2400 (LWP 3046)] 0x00007ffff6e989eb in __gconv_load_cache () at gconv_cache.c:62 62 gconv_cache.c: Datei oder Verzeichnis nicht gefunden. (gdb) info inferiors Num Description Executable 1 <null> /usr/bin/apt 3 <null> /usr/bin/dpkg 5 <null> /usr/bin/dpkg 6 <null> /usr/bin/apt * 7 process 3046 /usr/lib/apt/methods/https (gdb) cont Continuing. Thread 7.1 "https" hit Breakpoint 1, main (argv=<optimized out>) at ./methods/https.cc:546 546 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden. (gdb) bt #0 main (argv=<optimized out>) at ./methods/https.cc:546 (gdb) stepi 546 in ./methods/https.cc (gdb) HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:536 536 in ./methods/https.cc (gdb) disassemble HttpsMethod::~HttpsMethod Dump of assembler code for function HttpsMethod::~HttpsMethod(): 0x0000555555558f90 <+0>: lea 0x210889(%rip),%rax # 0x555555769820 <_ZTV11HttpsMethod+16> 0x0000555555558f97 <+7>: push %r12 0x0000555555558f99 <+9>: push %rbp 0x0000555555558f9a <+10>: mov %rdi,%rbp 0x0000555555558f9d <+13>: push %rbx 0x0000555555558f9e <+14>: mov %rax,(%rdi) 0x0000555555558fa1 <+17>: mov 0x100(%rdi),%rdi 0x0000555555558fa8 <+24>: callq 0x555555558398 0x0000555555558fad <+29>: lea 0x21099c(%rip),%rax # 0x555555769950 <_ZTV14BaseHttpMethod+16> 0x0000555555558fb4 <+36>: mov 0xd0(%rbp),%rdi ... 0x000055555555916d <+477>: jmpq 0x5555555590e4 <HttpsMethod::~HttpsMethod()+340> End of assembler dump. (gdb) stepi std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_dispose (this=<optimized out>) at /usr/include/c++/6/bits/basic_string.h:180 180 /usr/include/c++/6/bits/basic_string.h: Datei oder Verzeichnis nicht gefunden. (gdb) display/i $pc 1: x/i $pc => 0x55555555873d <main(int, char const**)+173>: add $0x10,%rbp (gdb) nexti HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:536 536 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden. 1: x/i $pc => 0x555555558741 <main(int, char const**)+177>: mov %rax,0x20(%rsp) (gdb) 538 in ./methods/https.cc 1: x/i $pc => 0x555555558746 <main(int, char const**)+182>: callq 0x555555558398 (gdb) stepi 0x0000555555558398 in ?? () 1: x/i $pc => 0x555555558398: jmpq *0x21193a(%rip) # 0x555555769cd8 (gdb) curl_easy_cleanup (data=0x55555579f6c0) at easy.c:829 829 easy.c: Datei oder Verzeichnis nicht gefunden. 1: x/i $pc => 0x7ffff77d9660 <curl_easy_cleanup>: test %rdi,%rdi (gdb) bt #0 curl_easy_cleanup (data=0x55555579f6c0) at easy.c:829 #1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538 #2 main (argv=<optimized out>) at ./methods/https.cc:546 (gdb) disassemble curl_easy_cleanup Dump of assembler code for function curl_easy_cleanup: => 0x00007ffff77d9660 <+0>: test %rdi,%rdi 0x00007ffff77d9663 <+3>: je 0x7ffff77d9670 <curl_easy_cleanup+16> 0x00007ffff77d9665 <+5>: jmpq 0x7ffff77cd4c0 <Curl_close> 0x00007ffff77d966a <+10>: nopw 0x0(%rax,%rax,1) 0x00007ffff77d9670 <+16>: repz retq End of assembler dump. (gdb) stepi 0x00007ffff77d9663 829 in easy.c 1: x/i $pc => 0x7ffff77d9663 <curl_easy_cleanup+3>: je 0x7ffff77d9670 <curl_easy_cleanup+16> (gdb) 833 in easy.c 1: x/i $pc => 0x7ffff77d9665 <curl_easy_cleanup+5>: jmpq 0x7ffff77cd4c0 <Curl_close> (gdb) Curl_close (data=0x55555579f6c0) at url.c:394 394 url.c: Datei oder Verzeichnis nicht gefunden. 1: x/i $pc => 0x7ffff77cd4c0 <Curl_close>: test %rdi,%rdi (gdb) disassemble Curl_close Dump of assembler code for function Curl_close: => 0x00007ffff77cd4c0 <+0>: test %rdi,%rdi 0x00007ffff77cd4c3 <+3>: je 0x7ffff77cd700 <Curl_close+576> 0x00007ffff77cd4c9 <+9>: push %rbp 0x00007ffff77cd4ca <+10>: push %rbx 0x00007ffff77cd4cb <+11>: mov %rdi,%rbx 0x00007ffff77cd4ce <+14>: sub $0x8,%rsp 0x00007ffff77cd4d2 <+18>: callq 0x7ffff77e15c0 <Curl_expire_clear> 0x00007ffff77cd4d7 <+23>: mov 0x60(%rbx),%rdi <<<<<<<<< $rbx contains for OP an invalid pointer 0x00007ffff77cd4db <+27>: test %rdi,%rdi 0x00007ffff77cd4de <+30>: je 0x7ffff77cd4e8 <Curl_close+40> 0x00007ffff77cd4e0 <+32>: mov %rbx,%rsi ... 0x00007ffff77cd702 <+578>: retq End of assembler dump. (gdb) b *0x00007ffff77cd4d7 Breakpoint 2 at 0x7ffff77cd4d7: file url.c, line 399. (gdb) cont Continuing. Thread 7.1 "https" hit Breakpoint 2, Curl_close (data=0x55555579f6c0) at url.c:399 399 in url.c 1: x/i $pc => 0x7ffff77cd4d7 <Curl_close+23>: mov 0x60(%rbx),%rdi (gdb) bt #0 Curl_close (data=0x55555579f6c0) at url.c:399 #1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538 #2 main (argv=<optimized out>) at ./methods/https.cc:546 (gdb) print/x $rip $1 = 0x7ffff77cd4d7 (gdb) bt full #0 Curl_close (data=0x55555579f6c0) at url.c:399 m = <optimized out> #1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538 No locals. #2 main (argv=<optimized out>) at ./methods/https.cc:546 Binary = "https" (gdb) down #1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538 538 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden. (gdb) down #0 Curl_close (data=0x55555579f6c0) at url.c:399 399 url.c: Datei oder Verzeichnis nicht gefunden. (gdb) print data $15 = (struct Curl_easy *) 0x55555579f6c0 (gdb) print data->multi $16 = (struct Curl_multi *) 0x0 (gdb) print &(data->multi) $17 = (struct Curl_multi **) 0x55555579f720 (gdb) print/x 0x55555579f720 - 0x55555579f6c0 $18 = 0x60 (gdb) print/x $rbx $19 = 0x55555579f6c0 (gdb) up #1 0x000055555555874b in HttpsMethod::~HttpsMethod (this=0x7fffffffe470, __in_chrg=<optimized out>) at ./methods/https.cc:538 538 ./methods/https.cc: Datei oder Verzeichnis nicht gefunden. (gdb) print curl $20 = (CURL *) 0x55555579f6c0 https://sources.debian.org/src/curl/7.52.1-5+deb9u10/lib/url.c/#L399 https://sources.debian.org/src/apt/1.4.10/methods/https.cc/#L538 https://github.com/curl/curl/issues/2764 https://github.com/curl/curl/commit/bc36c4f5463db6ddb098fe83ec7ae8ead012d01b
