Hi Anton,thanks for the details bug report and going the extra mile to verify it is still reproducible with testing.
I quickly checked the upstream bug tracker, and found https://github.com/rsyslog/rsyslog/issues/4302 https://github.com/rsyslog/rsyslog/issues/4175 https://github.com/rsyslog/rsyslog/issues/3915 which look related to your problem. All issues with relp+gnutlsAll of them point to an issue with GnuTLS and the bug reporters mentioned that switching to OpenSSL fixed the issue.
The Debian package is built with OpenSSL support. Would you mind testing if switching to OpenSSL fixes the issue for you? Regards, Michael On 10.11.21 16:22, Anton Khirnov wrote:
Package: rsyslog Version: 8.2102.0-2 Severity: important Dear Maintainer, since upgrading from buster to bullseye, rsyslog on some of my machines randomly exits. My setup contains - a centralized log server, receiving logs with imrelp - two relays, that forward logs from other hosts using imrelp+omrelp - multiple hosts that send their logs (to either one of the relays or directly to the central server) with omrelp I tried running rsyslog with debugging messages under gdb, right before it exits, the output is 1778.966318176:imrelp.c : imrelp.c: librelp: done epoll_wait, nEvents:1 1778.966486859:imrelp.c : imrelp.c: librelp: generic error: ecode 10014, emsg 'TLS record reception failed [gnutls error -54: Error in the pull function.]' 1778.966543315:imrelp.c : errmsg.c: Called LogMsg, msg: imrelp[10514]: error 'TLS record reception failed [gnutls error -54: Error in the pull function.]', object 'lstn 10514: conn to clt 2a00:c500:561:201:7910:b4d 8:2065:6c6b/2a00:c500:561:201:7910:b4d8:2065:6c6b' - input may not work as intended 1778.966585565:imrelp.c : operatingstate.c: osf: MSG imrelp[10514]: error 'TLS record reception failed [gnutls error -54: Error in the pull function.]', object 'lstn 10514: conn to clt 2a00:c500:561:201:7910:b4d8:2 065:6c6b/2a00:c500:561:201:7910:b4d8:2065:6c6b' - input may not work as intended: signaling new internal message via SIGTTOU: 'imrelp[10514]: error 'TLS record reception failed [gnutls error -54: Error in the pull functio n.]', object 'lstn 10514: conn to clt 2a00:c500:561:201:7910:b4d8:2065:6c6b/2a00:c500:561:201:7910:b4d8:2065:6c6b' - input may not work as intended [v8.2102.0 try https://www.rsyslog.com/e/2353 ]' rsyslogd: imrelp[10514]: error 'TLS record reception failed [gnutls error -54: Error in the pull function.]', object 'lstn 10514: conn to clt 2a00:c500:561:201:7910:b4d8:2065:6c6b/2a00:c500:561:201:7910:b4d8:2065:6c6b' - input may not work as intended [v8.2102.0 try https://www.rsyslog.com/e/2353 ] Cannot find user-level thread for LWP 25226: generic error (gdb) [Thread 0x7ffff536c700 (LWP 25237) exited] [Thread 0x7ffff5b6d700 (LWP 25236) exited] [Thread 0x7ffff676f700 (LWP 25234) exited] [Thread 0x7ffff6b70700 (LWP 25233) exited] [Thread 0x7ffff6f71700 (LWP 25232) exited] [Thread 0x7ffff7a6d240 (LWP 25226) exited] [Inferior 1 (process 25226) exited with code 01] I have observed the issue only on the central log server and both of the relays, not on the hosts -- i.e. only those systems that use imrelp. I can also reproduce it semi-reliably by SIGKILLing (so it doesn't close the connection cleanly) a RELP client (i.e. an rsyslog instance using omrelp), which will almost (but not quite) always cause its corresponding RELP server to exit in the above manner. So if I SIGKILL rsyslogd on one of the hosts, it will cause its relay to die, which in turn causes the central server to die, which in turn makes me very unhappy. Since logging is critical for my infrastructure, I would very much appreciate it if this was fixed promptly. rsyslog 8.2110.0-1 from testing still exhibits the issue. Cheers,
OpenPGP_signature
Description: OpenPGP digital signature
