Repository: spark Updated Branches: refs/heads/branch-1.0 2844fbbe7 -> febec742c
[SPARK-1394] Remove SIGCHLD handler in worker subprocess It should not be the responsibility of the worker subprocess, which does not intentionally fork, to try and cleanup child processes. Doing so is complex and interferes with operations such as platform.system(). If it is desirable to have tighter control over subprocesses, then namespaces should be used and it should be the manager's resposibility to handle cleanup. Author: Matthew Farrellee <[email protected]> Closes #1247 from mattf/SPARK-1394 and squashes the following commits: c36f308 [Matthew Farrellee] [SPARK-1394] Remove SIGCHLD handler in worker subprocess (cherry picked from commit 3c104c79d24425786cec0034f269ba19cf465b31) Signed-off-by: Aaron Davidson <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/febec742 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/febec742 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/febec742 Branch: refs/heads/branch-1.0 Commit: febec742c82852c736fe09110667e82092f2e5f9 Parents: 2844fbb Author: Matthew Farrellee <[email protected]> Authored: Sat Jun 28 18:39:27 2014 -0700 Committer: Aaron Davidson <[email protected]> Committed: Sat Jun 28 18:39:48 2014 -0700 ---------------------------------------------------------------------- python/pyspark/daemon.py | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/febec742/python/pyspark/daemon.py ---------------------------------------------------------------------- diff --git a/python/pyspark/daemon.py b/python/pyspark/daemon.py index b2f226a..5eb1c63 100644 --- a/python/pyspark/daemon.py +++ b/python/pyspark/daemon.py @@ -103,6 +103,7 @@ def worker(listen_sock): if os.fork() == 0: # Leave the worker pool signal.signal(SIGHUP, SIG_DFL) + signal.signal(SIGCHLD, SIG_DFL) listen_sock.close() # Read the socket using fdopen instead of socket.makefile() because the latter # seems to be very slow; note that we need to dup() the file descriptor because
