New submission from Selim Belhaouane :
Cannot clear signal handler set with loop.add_signal_handler in forked process
with signal.signal
# Context
I'm running an async web server with uvicorn[1] and have background processes
(using multiprocessing) doing CPU-bound work. uvicorn install signal handlers
with loop.add*signal_handler, if available[2]. It also implements the common
"attempt graceful shutdown on the first signal, and forcefully shutdown on the
second (or more) signal" pattern. The problem I noticed was: when \_forking* at
least one process, the server never gracefully shuts down, **even if I install
new signal handlers in the subprocesses**.
# Problem
Signal handlers installed with loop.add*signal_handler cannot be cleared in
forked processes with `signal.signal` (see Experiment 1), \_unless* using
`signal.SIG_DFL` or `signal.SIG_IGN` in the forked processes (see Experiment 3).
For the record:
- When installing signal handlers in the parent process with signal.signal,
this is not a problem (see Experiment 2).
- When using multiprocessing with the "spawn" method, this is not a problem.
Unsurprising, since signal handlers are not inherited from the parent process
when using "spawn" (see Experiment 4).
- When install signal handlers in the child proceess with
loop.add_signal_handler, this is not a problem (see Experiment 5).
# Experiments
You'll find a minimal `exp.py` file in the attached archive. There's a few
tweakable parameters at the top. I just run this with `python3.X exp.py`, wait
a second or two, and hit Ctrl+C.
`results.txt` shows the results of a few experiments with different set of
parameters.
Note that the point at which I hit Ctrl+C is indicated by the ♥ (heart) symbol,
due to some terminal weirdness, although this is quite useful in this case!
The subsections below detail what happens when Ctrl+C is hit.
You guys probably know this already, but Ctrl+C basically sends SIGINT to all
processes in the process tree.
## Experiment 1
- Installs a signal handler in the parent process with loop.add_signal_handler
- Starts 3 subprocesses with "fork"
- The subprocesses install a new signal handler with signal.signal
Outcome: both the parent handler and child handlers get called (3 calls to
handle_sig_worker, 4 calls to handle_sig_main)
**Expected**: I would expect a single call to handle_sig_main, and 3 calls to
handle_sig_worker, as in Experiment 2 (which uses signal.signal instead of
loop.add_signal_handler) or Experiment 4 (which uses "spawn" instead of "fork")
or Experiment 5 ()
## Experiment 2
As Experiment 1, but signal handlers are installed in the parent process with
signal.signal
Outcome: the parent handler gets called once, and the child handlers get called
3 times
## Experiment 3
Same as Experiment 1, but this time using `signal.SIG_DFL` as the callback in
the child processes.
Outcome: the child processes immediately terminate, thanks to SIG_DFL, and the
parent handler gets called only once!
## Experiment 4
Same as Experiment 1, but this time using "spawn" as the multiprocessing start
method.
Outcome: same as Experiment 2: parent handlers gets called once, child handlers
get called 3 times
## Experiment 5
Same as Experiment 1, but this time installing signal handlers in the child
processes with loop.add_signal_handler.
Outcome: Same as Experiment 2.
# Environment
I was able to replicate the issue with python 3.7, 3.8, 3.9 and 3.10 (didn't
even try 3.6) on Ubuntu 20.04. I obtained python via `apt`.
[1]: https://www.uvicorn.org/
[2]:
https://github.com/encode/uvicorn/blob/61a6cabb4580e1c923df396eac264803f599412c/uvicorn/server.py#L281
--
components: asyncio
files: python-async-signal-bug.zip
messages: 385928
nosy: asvetlov, selimb, yselivanov
priority: normal
severity: normal
status: open
title: Impossible to override signal handler set with add_signal_handler in
forked process
type: behavior
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49774/python-async-signal-bug.zip
___
Python tracker
<https://bugs.python.org/issue43064>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
