Package: haproxy Version: 1.5.6-1 Severity: important Tags: patch -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi! Documenting an important bug in haproxy 1.5.6 which may segfault on reload: commit 44cf5450005c3bb822fce3c0971f515c470b678b Author: Willy Tarreau <[email protected]> Date: Wed Oct 22 19:06:31 2014 +0200 BUG/MAJOR: cli: explicitly call cli_release_handler() upon error Dmitry Sivachenko reported an embarrassing problem where haproxy would sometimes segfault upon reload. After careful analysis and code inspection, what happens is related to the "show sess" command on the CLI, and it is not limited to reload operations only. When a "show sess" is running, once the output buffer is full, the stats applet grabs a reference to the session being dumped in order for the current pointer to be able to advance by itself should this session disappear while the buffer is full. The applet also uses a release handler that is called when the applet terminates to release such references. The problem is that upon error, the command line parser sets the applet state to STAT_CLI_O_END indicating it wants to terminate the processing. Unfortunately, the release handler which is called later to clean everything up relies on the applet's state to know what operations were in progress, and as such it does not release the reference. A later "show sess" or the completion of the task being watched lead to a LIST_DEL() on the task's list which point to a location that does not match the applet's reference list anymore and the process dies. One solution to this would be to add a flag to the current applet's state mentionning it must leave, without affecting the state indicating the current operation. It's a bit invasive but could be the long term solution. The short term solution simply consists in calling the release handler just before changing the state to STAT_CLI_O_END. That way everything that must be released is released in time. Note that the probability to encounter this issue is very low. It requires a lot of "show sess" or "show sess all" calls, and that one of them dies before being completed. That can happen if "show sess" is run in scripts which truncate the output (eg: "echo show sess|socat|head"). This could be the worst case as it almost ensures that haproxy fills a buffer, grabs a reference and detects the error on the socket. There's no config-based workaround to this issue, except refraining from issuing "show sess" on large connection counts or "show sess all". If that's not possible to block everyone, restricting permissions on the stats socket ensures only authorized tools can connect. This fix must be backported to 1.5 and to 1.4 (with some changes in 1.4 since the release function does not exist so the LIST_DEL sequence must be open-coded). Special thanks to Dmitry for the fairly complete report. diff --git a/src/dumpstats.c b/src/dumpstats.c index ebf66ecac55b..26b0a9f1d7a7 100644 - --- a/src/dumpstats.c +++ b/src/dumpstats.c @@ -131,6 +131,7 @@ static int stats_dump_stat_to_buffer(struct stream_interface *si, struct uri_aut static int stats_pats_list(struct stream_interface *si); static int stats_pat_list(struct stream_interface *si); static int stats_map_lookup(struct stream_interface *si); +static void cli_release_handler(struct stream_interface *si); /* * cli_io_handler() @@ -2336,6 +2337,7 @@ static void cli_io_handler(struct stream_interface *si) } else { /* output functions: first check if the output buffer is closed then abort */ if (res->flags & (CF_SHUTR_NOW|CF_SHUTR)) { + cli_release_handler(si); appctx->st0 = STAT_CLI_END; continue; } @@ -2389,6 +2391,7 @@ static void cli_io_handler(struct stream_interface *si) appctx->st0 = STAT_CLI_PROMPT; break; default: /* abnormal state */ + cli_release_handler(si); appctx->st0 = STAT_CLI_PROMPT; break; } - -- System Information: Debian Release: jessie/sid APT prefers unstable APT policy: (500, 'unstable'), (101, 'experimental') Architecture: amd64 (x86_64) Foreign Architectures: i386 Kernel: Linux 3.16-3-amd64 (SMP w/4 CPU cores) Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages haproxy depends on: ii adduser 3.113+nmu3 ii init-system-helpers 1.21 ii libc6 2.19-12 ii libpcre3 1:8.35-3.1 ii libssl1.0.0 1.0.1j-1 ii zlib1g 1:1.2.8.dfsg-2 haproxy recommends no packages. Versions of packages haproxy suggests: ii haproxy-doc 1.5.6-1 pn vim-haproxy <none> - -- no debconf information -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBCAAGBQJUVTYUAAoJEJWkL+g1NSX5CRQP/Ri21ETlvlV7JECHCcFhUibe 7IA5kDSd0MeDrBqw5a0G8ijtrP5uXJkGsB7PAi5QE80bG9jRPVd/4BJvcqM59LhU +Uw3ESXG2mtDpN7ZQ3bWCocbMxv6X4yGKW99NCTFMMwGa5PirCAKGT2Rasp/Fy94 Ztoh6m99BtGZN//nYgtzoIydsYIozOlVn+7T/Gv3N7nzUeUrs1R4q7Wp5K+iIC/L V3lB8GcZm+Vo5hoHeMZrp7uNrI8JSbHblUQGw08Vxxd3rNZ3fvhEtL+g6MIjYOoX 1lIlnJDNKIgE8A+0XTArO35QYdCv/tk9erCVihCR5FFc3goM/659CZLmjoYqBqY3 Khn/qmk0dawPHVdYW6dUtIXsQwfD85dXQjayecK7zV1lCH8yrVTJTbLn0SjgXoOQ 7lWXQjOXYeHDWpCKgROASTvk+5be5+5MdQf2AqqDcRoHup8sr9zpf9QQr6hMi5IB LU1xsjh/H0SZsqj19gT0uTzKsXFUpaCMeWQSThZKXOsk+mOD/LX6l76pvkUOTX+o IahOpHer0XiW6Wreg2aX7jcdK7lA09H/0w7XzvnwZVjPLjrQv3I/TE3IsbnK7Eop OW9sO3jtn9KdU8hHn1ZnEgsrYetND+aqQDxLhEdFdoR6pX+MyiXrEBnF37Ani3DQ FuHqoOjTC4xNQ8mon7TK =ANYB -----END PGP SIGNATURE----- -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected]
