[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Antoine Rozo
Maybe it's more the `a ||= b` that acts like a `a = a or b` in Python? But then I don't think there is a confusion on Python side because languages with a || operator usually already has a simple | with a different meaning. Le jeu. 27 févr. 2020 à 00:28, Nick Coghlan a écrit : > > > > On Thu., 27

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Nick Coghlan
On Thu., 27 Feb. 2020, 2:03 am Guido van Rossum, wrote: > On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz > wrote: > >> In my experience, the expression `value |= other` is a common idiom across >> programming languages to provide a default for `value` if it is "unset". >> > > Interesting. Can

[Python-Dev] 3.7.7 schedule accelerated, cutoff now 2020-03-02

2020-02-26 Thread Ned Deily
https://discuss.python.org/t/3-7-7-schedule-accelerated-cutoff-now-2020-03-02/3511 -- Ned Deily n...@python.org -- [] ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
Done. Thanks, --Elad ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Antoine Pitrou
Hi Elad, Can you open an issue on https://bugs.python.org/ and post your findings there? I don't think it makes sense to continue discussing this on python-dev. (note that opening a bug doesn't mean it will be fixed quickly, but at least it's recorded somewhere instead of being lost in the mai

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
I believe that the problem is in logging/__init__.py, which registers an atfork() handler for re-initializing its lock. However, as part of this process it attempts to acquire the module lock, which has not been reinitialized and so still reflects the parent's state of the lock. --Elad

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Antoine Pitrou
Admittedly, the whole distutils spawn code should be rewritten to use subprocess. Regards Antoine. On Wed, 26 Feb 2020 17:52:53 - "Elad Lahav" wrote: > A change to posix_spawnp() fixes the problem for me: > > diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py > index c

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
A simple example that reproduces the hang (please keep in mind that I have very little experience writing Python code...): import os from concurrent.futures import ThreadPoolExecutor def new_process(arg): pid = os.fork() if pid == 0: exec_fn("/bin/true

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
A change to posix_spawnp() fixes the problem for me: diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py index ceb94945dc..cb69de4242 100644 --- a/Lib/distutils/spawn.py +++ b/Lib/distutils/spawn.py @@ -90,7 +90,7 @@ def _spawn_posix(cmd, search_path=1, verbose=0, dry

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
It's actually not clear to me what lock it is from the core file I took, as rlock_acquire() is called through a function pointer from method_vectorcall_VARARGS_KEYWORDS() (I posted the backtrace separately). My suspicion is that it doesn't fail on macOS because it may keep all of the semaphore'

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
Sorry, should have posted the backtrace from the beginning. It goes deeper than this, but the important part is in the child after fork(): #0 SyncSemWait () at /builds/workspace/710-SDP/build_x86_64/lib/c/kercalls/x86_64/SyncSemWait.S:37 #1 0x004bfa174ac6 in PyThread_acquire_lock_timed (l

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Guido van Rossum
Here's our current proposal for docs. Is there anything you'd like to add? https://github.com/python/cpython/pull/18659/files On Wed, Feb 26, 2020 at 8:12 AM David Mertz wrote: > I think the "if unset" behavior is well handled by collections.ChainMap. > But I do think that fact should be promine

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Antoine Pitrou
Hi, On Tue, 25 Feb 2020 11:48:44 - e2la...@gmail.com wrote: > Hello, > > I have successfully built Python 3.8.1 on QNX, but ran into a problem when > using 'make -j4'. The build process invariably hangs with multiple > single-threaded child processes stuck indefinitely waiting on semaphor

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread David Mertz
I think the "if unset" behavior is well handled by collections.ChainMap. But I do think that fact should be prominent in any documentation of the new dict Union operator. On Wed, Feb 26, 2020, 11:06 AM Guido van Rossum wrote: > On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz > wrote: > >> In m

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Guido van Rossum
What semaphore is the subprocess blocking on? I.e. where in the Python code (presumably of ThreadPoolExecutor) is that semaphore defined? Given your hypothesis of the cause of the problem, can you perhaps write a much simper program (which is simpler to debug and simpler to reason about) that exhi

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Guido van Rossum
On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz wrote: > In my experience, the expression `value |= other` is a common idiom across > programming languages to provide a default for `value` if it is "unset". > For a > container-type, I would expect this operation, informally spoken, to be > appli

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav
More information: The hang happens when building extensions, using the setup.py script. The script determines that the build is parallel (build_ext.py/build_extensions) and creates a thread pool. Each thread then executes a compilation job by fork()ing a compiler process. I don't see how it wor

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Claudio Jolowicz
First off, apologies for entering the discussion after it has formally ended. I'm not sure this has any relevance now that the PEP is accepted and the reference implementation merged. If not, sorry and feel free to ignore. "Leftmost Value (First-Seen) Wins" was rejected because > it is not clear

[Python-Dev] Re: Merging PRs without CLA signed

2020-02-26 Thread Kyle Stanley
> I remember seeing it too. It may have originally been in the tracker > instructions, but should definitely be in the devguide now. >From looking through the devguide for every instance of "CLA" and "trivial", there seems to be just one section that mentions anything regarding the triviality of