Elad created the issue:
"distutils.spawn should use subprocess (hang in parallel builds on QNX)"
https://bugs.python.org/issue39763
Victor
Le mer. 26 févr. 2020 à 20:22, Elad Lahav a écrit :
>
> Done.
>
> Thanks,
> --Elad
> ___
> Python-Dev mailing li
Done.
Thanks,
--Elad
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list
Hi Elad,
Can you open an issue on https://bugs.python.org/ and post your
findings there? I don't think it makes sense to continue discussing
this on python-dev.
(note that opening a bug doesn't mean it will be fixed quickly, but at
least it's recorded somewhere instead of being lost in the mai
I believe that the problem is in logging/__init__.py, which registers an
atfork() handler for re-initializing its lock. However, as part of this process
it attempts to acquire the module lock, which has not been reinitialized and so
still reflects the parent's state of the lock.
--Elad
Admittedly, the whole distutils spawn code should be rewritten to use
subprocess.
Regards
Antoine.
On Wed, 26 Feb 2020 17:52:53 -
"Elad Lahav" wrote:
> A change to posix_spawnp() fixes the problem for me:
>
> diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py
> index c
A simple example that reproduces the hang (please keep in mind that I have very
little experience writing Python code...):
import os
from concurrent.futures import ThreadPoolExecutor
def new_process(arg):
pid = os.fork()
if pid == 0:
exec_fn("/bin/true
A change to posix_spawnp() fixes the problem for me:
diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py
index ceb94945dc..cb69de4242 100644
--- a/Lib/distutils/spawn.py
+++ b/Lib/distutils/spawn.py
@@ -90,7 +90,7 @@ def _spawn_posix(cmd, search_path=1, verbose=0,
dry
It's actually not clear to me what lock it is from the core file I took, as
rlock_acquire() is called through a function pointer from
method_vectorcall_VARARGS_KEYWORDS() (I posted the backtrace separately).
My suspicion is that it doesn't fail on macOS because it may keep all of the
semaphore'
Sorry, should have posted the backtrace from the beginning. It goes deeper than
this, but the important part is in the child after fork():
#0 SyncSemWait () at
/builds/workspace/710-SDP/build_x86_64/lib/c/kercalls/x86_64/SyncSemWait.S:37
#1 0x004bfa174ac6 in PyThread_acquire_lock_timed
(l
Hi,
On Tue, 25 Feb 2020 11:48:44 -
e2la...@gmail.com wrote:
> Hello,
>
> I have successfully built Python 3.8.1 on QNX, but ran into a problem when
> using 'make -j4'. The build process invariably hangs with multiple
> single-threaded child processes stuck indefinitely waiting on semaphor
What semaphore is the subprocess blocking on? I.e. where in the Python code
(presumably of ThreadPoolExecutor) is that semaphore defined?
Given your hypothesis of the cause of the problem, can you perhaps write a
much simper program (which is simpler to debug and simpler to reason about)
that exhi
More information:
The hang happens when building extensions, using the setup.py script. The
script determines that the build is parallel (build_ext.py/build_extensions)
and creates a thread pool. Each thread then executes a compilation job by
fork()ing a compiler process.
I don't see how it wor
12 matches
Mail list logo
