Davin Potts added the comment:
In your first example (not sample.py), if I make the following change:
#queue = multiprocessing.Manager().Queue(-1)
queue = 42
I am still able to reproduce the exception you observe. This suggests the
issue has nothing to do with the use of multiprocessing but
Changes by Davin Potts :
--
nosy: +vinay.sajip
___
Python tracker
<http://bugs.python.org/issue29168>
___
___
Python-bugs-list mailing list
Unsubscribe:
Davin Potts added the comment:
Simon: You do have a lock object inside the logging handler object. That is
what prevents your pickling of the logging handler object.
The background information Serhiy shared is focused on why pickling a
_thread.RLock object is problematic and unsupportable
Davin Potts added the comment:
This arises from the behavior of pickle (which is used by default in
multiprocessing to serialize objects sent to / received from other processes in
data exchanges), as seen with Python 3.6:
>>> import pickle
>>> x = pickle.dumps(mock.sentinel
Davin Potts added the comment:
I think this should be regarded as a duplicate of issue20804 though discussion
in issue14577 is also related/relevant.
--
superseder: -> Sentinels identity lost when pickled (unittest.mock)
___
Python tracker
&l
Davin Potts added the comment:
Serhiy: The above discussion seemed to converge on the perspective that object
identity should not survive pickling and that the point of a sentinel is object
identity. While your proposed patch may mechanically work, I believe it is in
conflict with the
Davin Potts added the comment:
I'm having difficulty watching your video attachment. Would it be possible to
instead describe, preferably with example code that others can similarly try to
reproduce the behavior, what you're experiencing?
Please keep in mind what the documentation
Changes by Davin Potts :
--
nosy: +davin
___
Python tracker
<http://bugs.python.org/issue29284>
___
___
Python-bugs-list mailing list
Unsubscribe:
Davin Potts added the comment:
Though this issue is specifically concerned with runpy APIs and their impact
especially in running unittest test scripts, it's worth commenting here for
people who need a workaround in the short term: code such as that shared in
http://stackoverflow.
Changes by Davin Potts :
--
versions: +Python 2.7, Python 3.7
___
Python tracker
<http://bugs.python.org/issue19675>
___
___
Python-bugs-list mailing list
Unsub
Davin Potts added the comment:
@Winterflower: Thank you for encouraging @dsoprea to create the new PR and
working to convert the previous patch.
@dsoprea: Thank you for taking the time to create the PR especially after this
has been sitting unloved for so long.
Though the new workflow using
Davin Potts added the comment:
For triggering the exception, supplying a Process target that deliberately
fails sounds right.
As for tests for the various start methods (fork/forkserver/spawn), if you are
looking at the 3.x branches you'll find this was been consolidated so that one
Davin Potts added the comment:
When passing judgement on what is "too basic", the initial example should be so
basic as to be immediately digestible by as many people as possible.
Some background:
All too many examples mislead newcomers into believing that the number of
processes
Davin Potts added the comment:
It would be nice to find an appropriate place to document the solid general
guidance Raymond provided; though merely mentioning it somewhere in the docs
will not translate into it being noticed. Not sure where to put it just yet...
Martin: Is there a specific
Davin Potts added the comment:
I took the example snippet and cleaned things up a bit, adding some crude
timestamping and commentary. (Attached)
In the example, when the Process finally dies, a lot of information has been
put onto the Queue but it hasn't necessarily had enough time
Changes by Davin Potts :
--
nosy: +davin
___
Python tracker
<http://bugs.python.org/issue26903>
___
___
Python-bugs-list mailing list
Unsubscribe:
Davin Potts added the comment:
The spawned process (you appear to have run on Windows, so I'm assuming spawn
but that's not so significant) has triggered an unhandled exception. If the
triggering and subsequent sending of the traceback to stderr is synchronous and
completes bef
Davin Potts added the comment:
While I believe I understand the motivation behind the suggestion to detect
when the code is doing something potentially dangerous, I'll point out a few
things:
* any time you ask for a layer of convenience, you must choose something to
sacrifice to g
Davin Potts added the comment:
@r.david.murray: Oh man, I was not going to go as far as advocate dropping the
GIL. :)
At least not in situations like this where the exploitable parallelism is meant
to be at the Python level and not inside the Fortran code (or that was my
understanding of
Changes by Davin Potts :
--
stage: -> resolved
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue27422>
___
___
Python-bugs-list
Davin Potts added the comment:
It is a general rule that when a process terminates, it likewise terminates all
its threads (unless a thread has been explicitly detached). How it goes about
doing so is complicated.
Remember that POSIX threads have no concept of parent/child among themselves
Davin Potts added the comment:
Tim: Totally agreed about threading.Thread not being a POSIX thread. It was
not my intent to suggest that they were equivalent -- apologies for the
confusion.
Instead I was attempting to describe a mentality of processes and their common
behavior across
New submission from Davin Potts:
The introduction section of the multiprocessing module's documentation does not
adhere to the python dev guidelines for keeping things in the affirmative tone.
Problem description:
Specifically, the intro section contains a warning block that, while conv
Davin Potts added the comment:
Attached is a proposed patch for the 2.7 branch.
It provides 2 changes to the documentation:
1) Moves the warning text block regarding synchronization functionality not
necessarily being available on all systems to the "Synchronization between
processes&quo
Davin Potts added the comment:
Attached is a proposed patch for the 3.4 branch.
The same modifications are made in this patch as in the patch for the 2.7
branch, with two minor tweaks:
1) The affirmative tone example uses Pool in a with statement as is supported
since 3.3.
2) References in
Changes by Davin Potts :
--
nosy: +rhettinger
___
Python tracker
<http://bugs.python.org/issue22952>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Davin Potts:
The organization and multiple sections of the multiprocessing module's
documentation do not adhere to the Python Dev Guidelines for economy of
expression, affirmative tone, and/or code examples.
Problem description:
The raw material and information i
Davin Potts added the comment:
I am happy to provide proposed patches but first can someone please clarify for
me whether I should have those patches depend upon the patches from Issue 22952?
--
___
Python tracker
<http://bugs.python.org/issue23
Davin Potts added the comment:
Procedural question (wanting to understand the label given to this issue):
when documentation does not adhere to the Python Developer's Guide, is a fix
for this considered an "enhancement"?
--
versio
Changes by Davin Potts :
--
nosy: +davin
___
Python tracker
<http://bugs.python.org/issue23051>
___
___
Python-bugs-list mailing list
Unsubscribe:
Davin Potts added the comment:
Successfully reproduced the behavior playing through variations with the
supplied examples (very helpful) on OS X v10.9.5 both in 2.7.9 and the default
branch -- adding version 3.5 to issue.
Also successfully verified that the supplied patches do just as
Davin Potts added the comment:
On Windows 7 (64-bit), it was not possible to reproduce any infinite looping
behavior with the supplied example code.
Specifically, with the two examples from Benjamin, the observed behavior when
running them was the same under 2.7.8 and default (3.5): a
Davin Potts added the comment:
Having reviewed the code commits made by Richard more than 2.5 years ago in
conjunction with his related code commits from already-closed Issue #15064, I
am led to conclude that all the items described in this issue have been
addressed and implemented
Davin Potts added the comment:
At nearly 5 years of age, this issue unfortunately never saw a viable
demonstration of how to provoke the reported behavior. Hopefully the reporter
either (1) discovered the nature of the problem originated elsewhere and he was
able to quickly fix it, or (2
Changes by Davin Potts :
--
nosy: +davin
___
Python tracker
<http://bugs.python.org/issue18620>
___
___
Python-bugs-list mailing list
Unsubscribe:
Davin Potts added the comment:
The example demonstrating the issue is reproducible on Windows (tested on
Windows 7 64-bit, specifically) with 2.7.9. Complications arising from how
multiprocessing creates new processes on Windows combined with conventions in
the import system in 2.7.9 result
Davin Potts added the comment:
Attached are proposed patches for 2.7, 3.4, and default (3.5) branches.
In these patches, the 2 examples in the documentation that showcase the use of
pool.apply_async have been modified to specifically highlight that a single
invocation of apply_async will only
Changes by Davin Potts :
Added file: http://bugs.python.org/file38031/issue18620_py27.patch
___
Python tracker
<http://bugs.python.org/issue18620>
___
___
Python-bug
Changes by Davin Potts :
Added file: http://bugs.python.org/file38030/issue18620_py34.patch
___
Python tracker
<http://bugs.python.org/issue18620>
___
___
Python-bug
Davin Potts added the comment:
Having installed a fresh copy of Debian Hurd into a VM, I am able to reproduce
the described issue using this 2-line snippet of code:
import multiprocessing
q = multiprocessing.Queue()
It was possible to reproduce the issue both using the builds of
Davin Potts added the comment:
Apologies -- it was already pointed out that there is no sem_open
implementation on Hurd.
--
___
Python tracker
<http://bugs.python.org/issue23
Davin Potts added the comment:
The provided links to the relevant code are unfortunately pointing at the
master branch's copy of those files and not a specific version of the code
making it that much harder to now discern where the issue arises. (A number of
things have changed in the m
Davin Potts added the comment:
It's a little unclear if one or more of the following is true:
(1) you are having trouble getting the multiprocessing module to build properly
on Solaris 2.8;
(2) you are having trouble getting your own code to build in a similar way to
the multiprocessing m
Davin Potts added the comment:
Thank you for the provided test case but because it depends upon compiled code
(the libsvm.so.2 file you supplied) it:
(1) makes me wonder if the issue might not arise from an issue inside the
supplied library (perhaps it was not rebuilt properly on your Ubuntu
Davin Potts added the comment:
Attached are proposed patches for default (3.5), 3.4, and 2.7 branches. (The
patch for 3.4 is identical to that for 3.5 so there are only two files in total
being attached.)
Regarding the exception being raised:
* An ImportError is now consistently being raised
Davin Potts added the comment:
To be clear, the changes to 2.7 are exclusively in the documentation. Changes
to 3.4 and default (3.5) are in both documentation and code.
--
Added file: http://bugs.python.org/file38078/issue23400_py27.patch
Davin Potts added the comment:
The points made by Travis are clear and solid.
Closing as this functionality is already handled well and no exceptional
situations are being argued for that would require a special case.
--
nosy: +davin
resolution: -> rejected
stage: -> resolved
Changes by Davin Potts :
--
stage: needs patch -> resolved
___
Python tracker
<http://bugs.python.org/issue8094>
___
___
Python-bugs-list mailing list
Un
Changes by Davin Potts :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue12738>
___
___
Python-bugs-list mailing list
Unsubscrib
Changes by Davin Potts :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue23072>
___
___
Python-bugs-list mailing list
Unsubscrib
Changes by Davin Potts :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue21429>
___
___
Python-bugs-list mailing list
Unsubscrib
Davin Potts added the comment:
Closing on the basis that:
1. Richard provided a proposed workaround but did not receive a response from
the OP in 1.5 years.
2. Issues encountered by the OP may have been addressed in part by changes in
the way import works in the past 4.5 years.
3. This issue
Changes by Davin Potts :
--
resolution: -> out of date
stage: -> resolved
status: pending -> closed
___
Python tracker
<http://bugs.python.o
Davin Potts added the comment:
In some (but not necessarily all) circumstances, the multiprocessing module is
now (in 2.7 and 3.x) able to detect the "infinite spawning" behavior described
due to unsafe importing of the main module on Windows. The resulting error
message looks
Davin Potts added the comment:
Closing per the feedback of the OP and secondary contributor to the issue.
Kudos to the reporters for such nice follow-up.
--
nosy: +davin
resolution: -> not a bug
stage: -> resolved
status: open -&g
Davin Potts added the comment:
Closing this issue after having verified that the issue can no longer be
reproduced on the systems mentioned (Ubuntu 10.04 or OSX). Related issues such
as issue9205 have been addressed elsewhere and other possibly related issues
such as issue22393 are being
Davin Potts added the comment:
Adding Brett Cannon as this issue appears to really be about doing an import
shortly after an os.fork() -- this may be of particular interest to him.
This issue probably should have had Brett and/or others added to nosy long ago.
--
nosy: +brett.cannon
Davin Potts added the comment:
This issue was marked as "not a bug" by OP a while back but for whatever reason
it did not also get marked as "closed". Going ahead with closing it now.
--
nosy: +davin
stage: needs patch -> resolved
Davin Potts added the comment:
Though the code may have changed a bit in the meantime (Issue11750 in
particular), the calls to _winapi.SetNamedPipeHandleState in
Lib/multiprocessing/connection.py are still present and largely the same as
when this issue was first opened.
The implementation
Changes by Davin Potts :
Added file: http://bugs.python.org/file38175/issue23400_py27_improveddocs.patch
___
Python tracker
<http://bugs.python.org/issue23400>
___
___
Davin Potts added the comment:
Attaching revised patches with improved phrasing in the changes to the docs.
Thanks goes to Berker for reviewing and providing helpful feedback, especially
paying close attention to detail in the docs.
--
Added file:
http://bugs.python.org/file38174
Davin Potts added the comment:
There are at least two issues at play here.
Running the attached file on OS X produces starkly different results -- console
prints:
CREATED TEMP DIRECTORY
/var/folders/s4/tc1y5rjx25vfknpzvnfh1b14gn/T/temp_z6I0BA
task1
task2
ATEXIT: REMOVING TEMP DIRECTORY
Davin Potts added the comment:
I should have added in my prior comments:
juj: thank you very much for providing the info about the platform you tested
on and even an example piece of code that triggered the problem. I wish all
issues came with the level of info you provided
Davin Potts added the comment:
Interesting! The documentation in 3.4 as well as 2.7 indicates that the
keyword should be 'blocking' yet the code implements this as 'block'.
Code to reproduce empirically what is actually implemented:
import multiprocessing
dummy_lock =
Davin Potts added the comment:
You make an overall valid point that despite reading the documentation, the
resulting behavior of your code was not what you expected -- I take that
specific complaint very seriously anytime anyone makes it.
Regarding your recommendations:
I) Unfortunately
Davin Potts added the comment:
Of course, there's code in the wild that expects and uses the parameter named
'block' so simply changing this keyword will result in breaking others' code.
Two potentially appealing options:
1) Document that acquire in multiprocessing diffe
Davin Potts added the comment:
At first blush it does appear there is potential for conflict because of how
the semaphore filename template was implemented -- that's a cool find. In
practice, I wonder how often this has actually bitten anyone in the real world.
The Linux world
Davin Potts added the comment:
Without further information from the OP, there is not much more we can do as
the problem may well originate with the user's compiled C code (which
circumstantially seems quite likely). Closing as out-of-date.
--
resolution: -> out of dat
Davin Potts added the comment:
Triggering it regularly in a build farm indeed sounds like genuine pain.
@Paul or @vapier: In tracking down this issue, did you already create a
convenient way to repro the misbehavior that could be used in testing? Any
finalized patch that we make will need
Davin Potts added the comment:
Closing as there is no additional information forthcoming from the OP and per
the prior comments, Torsten's suggestion seems as helpful as anything that can
be offered without more info.
--
resolution: -> out of date
status: pending -
Changes by Davin Potts :
--
type: -> enhancement
___
Python tracker
<http://bugs.python.org/issue24475>
___
___
Python-bugs-list mailing list
Unsubscrib
Davin Potts added the comment:
The 'crash' type is reserved for core dump type situations; 'behavior' is more
appropriate for the misbehavior your describe.
--
type: crash -> behavior
___
Python tracker
<http://
Davin Potts added the comment:
@neologix: Budgeting time this week to have a proper look -- sorry I haven't
gotten back to it sooner.
--
___
Python tracker
<http://bugs.python.org/is
Davin Potts added the comment:
I have been able to reproduce the behavior you described under 3.5 under one
set of circumstances so far.
When attempting to use classes/functions/other not defined in an importable
module, an AttributeError is expected as you observed. The viability of that
Changes by Davin Potts :
--
status: open -> pending
versions: +Python 3.5
___
Python tracker
<http://bugs.python.org/issue25053>
___
___
Python-bugs-list mai
New submission from Davin Potts:
Inspired by issue24391 and the changes proposed to the threading module's reprs
for Event, Semaphore, BoundedSemaphore, and Barrier, the corresponding objects
in the multiprocessing module should have their reprs updated accordingly.
--
assignee:
Changes by Davin Potts :
--
dependencies: +Better repr for threading objects
___
Python tracker
<http://bugs.python.org/issue25066>
___
___
Python-bugs-list mailin
Davin Potts added the comment:
This patch implements the majority opinion from issue24391 for the desired
format of the reprs produced by Event, Semaphore, BoundedSemaphore, and
Barrier. It provides tests around each, inspired by Serhiy's preliminary patch
for that same issue but adapte
Davin Potts added the comment:
Echoing Larry's insightful summary, it would not be particularly rewarding to
pursue a goal of making the whole world of reprs consistent. But as Antoine
and Serhiy began to point out, the multiprocessing module not only has its own
Event, Sema
Davin Potts added the comment:
To Antoine's point in issue24391, here too, these modifications to the format
of the repr are estimated to have very little impact or risk to breaking
existing code.
--
___
Python tracker
<http://bugs.py
Davin Potts added the comment:
@berker.peksag, @r.david.murray: Could I gently nudge one of you into
committing the final versions of these patches?
--
___
Python tracker
<http://bugs.python.org/issue23
Davin Potts added the comment:
The patches make good sense to me -- I have no comments to add in a review.
I spent more time than I care to admit concerned with the idea that
error_callback (exposed by map_async which map sits on top of) should perhaps
be called not just once at the end but
Changes by Davin Potts :
--
stage: needs patch -> patch review
___
Python tracker
<http://bugs.python.org/issue23992>
___
___
Python-bugs-list mailing list
Un
Davin Potts added the comment:
If I understand your motivations correctly, I'd restate them as: you want a
mechanism for being immediately notified of an exception in a parent process
without waiting on any child processes of that parent to finish and furthermore
propose this should b
Changes by Davin Potts :
--
nosy: +jnoller, sbt
___
Python tracker
<http://bugs.python.org/issue24948>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Davin Potts :
--
type: -> behavior
___
Python tracker
<http://bugs.python.org/issue24948>
___
___
Python-bugs-list mailing list
Unsubscrib
Davin Potts added the comment:
As an aside: issue24948 seems to show there are others who would find the
immediate-multiple-error_callback idea attractive.
--
___
Python tracker
<http://bugs.python.org/issue23
Changes by Davin Potts :
--
stage: -> patch review
type: -> enhancement
___
Python tracker
<http://bugs.python.org/issue25169>
___
___
Python-bugs-list
Davin Potts added the comment:
The proposed patch would potentially break existing code which anticipates the
current behavior. The potential negative impact means this particular proposed
patch is not viable.
Choices forward include: (1) better documenting the existing, established
Changes by Davin Potts :
--
versions: +Python 3.6
___
Python tracker
<http://bugs.python.org/issue18620>
___
___
Python-bugs-list mailing list
Unsubscribe:
Davin Potts added the comment:
Good catch and agreed this should be cleaned up.
Patch looks good to me and given its nature is probably all we need although
it's probably worth combining this minor documentation update so that it gets
applied with some other documentation patch(es) a
Changes by Davin Potts :
--
title: multiprocess documentation -> multiprocessing docs example still refs
os.getppid as unix-only
___
Python tracker
<http://bugs.python.org/issu
Changes by Davin Potts :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue24153>
___
___
Python-bugs-list mailing list
Unsubscrib
Davin Potts added the comment:
This appears to be highly dependent upon particular versions of operating
systems and specific versions of various libraries, some built with certain
options. It does not appear to be an issue in multiprocessing itself.
@Ivan.K: Unless there is a clear example
Changes by Davin Potts :
--
status: open -> pending
___
Python tracker
<http://bugs.python.org/issue21345>
___
___
Python-bugs-list mailing list
Unsubscrib
Changes by Davin Potts :
--
nosy: +davin
___
Python tracker
<http://bugs.python.org/issue24427>
___
___
Python-bugs-list mailing list
Unsubscribe:
Changes by Davin Potts :
--
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue23979>
___
___
Python-bugs-list mailing list
Unsubscrib
Davin Potts added the comment:
Berker: It looks to me like the docs are indeed in sync with the code on Value
in that lock really is a keyword-only argument. It looks like it's been that
way since at least 2.7 (I didn't look at earlier). There are enough other
thi
Davin Potts added the comment:
Can you provide more relevant information to help others reproduce this? What
operating system, did you install Python yourself or did it come with the
operating system, what Python package you are trying to install (the package
supplying the setup.py you&#x
Changes by Davin Potts :
--
nosy: +davin
___
Python tracker
<http://bugs.python.org/issue12939>
___
___
Python-bugs-list mailing list
Unsubscribe:
101 - 200 of 351 matches
Mail list logo
