[issue23072] 2.7.9 multiprocessing compile conflict

2015-04-11 Thread Davin Potts
Davin Potts added the comment: Closing this very stale issue as out of date with no response from OP since request months ago for enough info to be able to proceed. -- resolution: -> out of date status: pending -> closed ___ Python tracker

[issue21429] Input.output error with multiprocessing

2015-04-11 Thread Davin Potts
Davin Potts added the comment: Closing this stale issue as out of date with no response from OP since request months ago for enough info to be able to proceed. -- resolution: -> out of date status: pending -> closed ___ Python tracker

[issue23979] Multiprocessing Pool.map pickles arguments passed to workers

2015-04-18 Thread Davin Potts
Davin Potts added the comment: Though it's been discussed elsewhere, issue17560 is a good one where the matter of "really big" objects are being communicated between processes via multiprocessing. In it, Richard shares some detail about the implementation in multiprocessing,

[issue23992] multiprocessing: MapResult shouldn't fail fast upon exception

2015-04-18 Thread Davin Potts
Davin Potts added the comment: This is a nice example demonstrating what I agree is a problem with the current implementation of close. A practical concern with what I believe is being proposed in your trivial fix: if the workers are engaged in very long-running tasks (and perhaps slowly

[issue23979] Multiprocessing Pool.map pickles arguments passed to workers

2015-04-22 Thread Davin Potts
Davin Potts added the comment: @Luis: Before closing this bug, could you give a quick description of the fix you put in place? Two reasons: 1) Someone else encountering this frustration with similar needs might well benefit from hearing what you did. 2) To provide evidence that there is

[issue23713] intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception

2015-04-22 Thread Davin Potts
Davin Potts added the comment: Serhiy: If my comments on your questions make sense, should we go ahead with the patch as it is? -- ___ Python tracker <http://bugs.python.org/issue23

[issue24033] Update _test_multiprocessing.py to use script helpers

2015-04-23 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue24033> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21791] Proper return status of os.WNOHANG is not always (0, 0)

2015-04-27 Thread Davin Potts
Davin Potts added the comment: The man pages for waitpid on OpenBSD 5.x do not suggest any meaningful value will be returned in status when WNOHANG is requested and no child processes have anything to report. The following session attempts to exercise os.waitpid using Python 2.7.9 on an

[issue21345] multiprocessing.Pool._handle_workers sleeps too long

2015-04-27 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue21345> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue9782] _multiprocessing.c warnings under 64-bit Windows

2015-04-27 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue9782> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22255] Multiprocessing freeze_support raises RuntimeError

2015-05-08 Thread Davin Potts
Davin Potts added the comment: The issue may already be addressed for the OP and without further information we don't know what or if anything needs pursuing. If new information surfaces, a new issue should be opened. -- resolution: -> out of date status: pending -

[issue21162] code in multiprocessing.pool freeze if inside some code from scikit-learn (and probably liblinear) executed on ubuntu 12.04 64 Bit

2015-05-08 Thread Davin Potts
Davin Potts added the comment: It is no longer possible to reproduce the described issue. Changes to the relevant libraries since this issue was originally opened appear to have addressed the cause. Going ahead with closing this issue as there have been no further reports from the OP or

[issue13044] pdb throws AttributeError at end of debugging session

2015-05-08 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue13044> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21162] code in multiprocessing.pool freeze if inside some code from scikit-learn (and probably liblinear) executed on ubuntu 12.04 64 Bit

2015-05-09 Thread Davin Potts
Davin Potts added the comment: @Ivan.K: Can you be more specific about which linux platform you are using? To reproduce, should I use Ubuntu, Fedora, OpenSUSE, ... and which version of that linux distro? -- ___ Python tracker <h

[issue24153] threading/multiprocessing tests fail on chromebook under crouton generated chroots

2015-05-10 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue24153> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue24202] Multiprocessing Pool not working for userdefined function

2015-05-15 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin type: crash -> behavior ___ Python tracker <http://bugs.python.org/issue24202> ___ ___ Python-bugs-list mailing list Un

[issue24033] Update _test_multiprocessing.py to use script helpers

2015-05-15 Thread Davin Potts
Changes by Davin Potts : -- stage: -> patch review type: -> enhancement ___ Python tracker <http://bugs.python.org/issue24033> ___ ___ Python-bugs-list

[issue6766] Cannot modify dictionaries inside dictionaries using Managers from multiprocessing

2016-09-06 Thread Davin Potts
Davin Potts added the comment: Attaching patch for default (3.6) branch which implements what was previously described and discussed, updates the documentation to explain this updated behavior, and includes new tests. @yselivanov: Can you think of any edge cases that should be handled but

[issue9400] multiprocessing.pool.AsyncResult.get() messes up exceptions

2016-09-06 Thread Davin Potts
Davin Potts added the comment: For clarity: * Confirmed, can repro on 2.7.12 on OSX. * Also confirmed that this can not be reproduced on the 3.5 or 3.6 branches. -- versions: -Python 3.5, Python 3.6 ___ Python tracker <http://bugs.python.

[issue9400] multiprocessing.pool.AsyncResult.get() messes up exceptions

2016-09-06 Thread Davin Potts
Davin Potts added the comment: Reading through issue1692335 provides a sense of the concerns surrounding the patches they applied to the 3.x branches. Attempting to backport those patches to the 2.7 branch involves non-trivial risk, magnified by the now numerous differences between those

[issue18844] allow weights in random.choice

2016-09-06 Thread Davin Potts
Davin Potts added the comment: I've gone through the patch -- looks good to me. -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue18844> ___ ___

[issue6766] Cannot modify dictionaries inside dictionaries using Managers from multiprocessing

2016-09-07 Thread Davin Potts
Davin Potts added the comment: Updating previously supplied patch for 3.6 to the right format. -- Added file: http://bugs.python.org/file8/issue_6766_py36.nogit.patch ___ Python tracker <http://bugs.python.org/issue6

[issue6766] Cannot modify dictionaries inside dictionaries using Managers from multiprocessing

2016-09-07 Thread Davin Potts
Davin Potts added the comment: Attaching updated patch to reflect Yury's suggested changes from review. -- Added file: http://bugs.python.org/file44453/issue_6766_py36.nogit.yuryfeedback.patch ___ Python tracker <http://bugs.python.org/i

[issue6766] Cannot modify dictionaries inside dictionaries using Managers from multiprocessing

2016-09-07 Thread Davin Potts
Davin Potts added the comment: Fixed in upcoming 3.6. -- resolution: -> fixed stage: patch review -> resolved ___ Python tracker <http://bugs.python.org/

[issue6721] Locks in the standard library should be sanitized on fork

2016-09-07 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue6721> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue21201] Uninformative error message in multiprocessing.Manager()

2016-09-08 Thread Davin Potts
Changes by Davin Potts : -- assignee: -> davin ___ Python tracker <http://bugs.python.org/issue21201> ___ ___ Python-bugs-list mailing list Unsubscrib

[issue21201] Uninformative error message in multiprocessing.Manager()

2016-09-08 Thread Davin Potts
Davin Potts added the comment: Looks good to me as well. I'll go ahead with applying the patch. Berker: I'm not sure it's worth adding something like `util.info("Serialization failure on: %r", msg)`. If serialization failed for some reason other than MemoryError,

[issue21201] Uninformative error message in multiprocessing.Manager()

2016-09-08 Thread Davin Potts
Davin Potts added the comment: Thanks for your patch @wojtekwalczak! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <http://bugs.python.or

[issue21009] Potential deadlock in concurrent futures when garbage collection occurs during Queue.get

2016-09-09 Thread Davin Potts
Davin Potts added the comment: Sounds related to issue14976. -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue21009> ___ ___ Python-bugs-list mailin

[issue21009] Potential deadlock in concurrent futures when garbage collection occurs during Queue.get

2016-09-09 Thread Davin Potts
Changes by Davin Potts : -- superseder: -> queue.Queue() is not reentrant, so signals and GC can cause deadlocks ___ Python tracker <http://bugs.python.org/issu

[issue21009] Potential deadlock in concurrent futures when garbage collection occurs during Queue.get

2016-09-09 Thread Davin Potts
Davin Potts added the comment: Per the decision of issue14976. -- resolution: -> wont fix status: open -> closed ___ Python tracker <http://bugs.python.org/i

[issue28052] clarify concurrent.futures docs to not refer to async Futures except where truly necessary

2016-09-09 Thread Davin Potts
New submission from Davin Potts: Per discussions with Yury, Lukasz and BDFL. -- messages: 275417 nosy: davin priority: normal severity: normal status: open title: clarify concurrent.futures docs to not refer to async Futures except where truly necessary type: enhancement versions

[issue9400] multiprocessing.pool.AsyncResult.get() messes up exceptions

2016-09-09 Thread Davin Potts
Davin Potts added the comment: Richard already backported the fix for objects which can not be pickled but the second fix (applied to 3.3 in #1692335) would be problematic to similarly backport to 2.7. Because there are reasonable workarounds for this in 2.7, going ahead with marking this

[issue21009] Potential deadlock in concurrent futures when garbage collection occurs during Queue.get

2016-09-09 Thread Davin Potts
Changes by Davin Potts : -- resolution: wont fix -> status: closed -> open ___ Python tracker <http://bugs.python.org/issue21009> ___ ___ Python-bugs-list

[issue28053] parameterize what serialization is used in multiprocessing

2016-09-09 Thread Davin Potts
New submission from Davin Potts: Currently multiprocessing uses the pickle module for its serialization of objects to be communicated between processes. Specifically v2 of the pickle protocols is now exclusively used to provide maximum compatibility, motivated by the desire for multiple

[issue28053] parameterize what serialization is used in multiprocessing

2016-09-09 Thread Davin Potts
Changes by Davin Potts : -- components: +Library (Lib) ___ Python tracker <http://bugs.python.org/issue28053> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue26507] Use highest pickle protocol in multiprocessing

2016-09-09 Thread Davin Potts
Davin Potts added the comment: Closing in deference to the enhancement described in issue28053. -- resolution: -> duplicate status: open -> closed superseder: -> parameterize what serialization is used in multiprocessing ___ Python track

[issue23403] Use pickle protocol 4 by default?

2016-09-09 Thread Davin Potts
Davin Potts added the comment: Closing in deference to the enhancement described in issue28053. -- nosy: +davin resolution: -> duplicate status: open -> closed ___ Python tracker <http://bugs.python.org/i

[issue23403] Use pickle protocol 4 by default?

2016-09-09 Thread Davin Potts
Changes by Davin Potts : -- superseder: -> parameterize what serialization is used in multiprocessing ___ Python tracker <http://bugs.python.org/issu

[issue28053] parameterize what serialization is used in multiprocessing

2016-09-09 Thread Davin Potts
Davin Potts added the comment: Attaching patch containing refactorizations but missing update to docs for the purposes of review. Introduces three new things: * a function to get the current serializer/reducer * a function to set the serializer/reducer * an abstract base class to assist others

[issue27873] multiprocessing.pool.Pool.map should take more than one iterable

2016-09-28 Thread Davin Potts
Davin Potts added the comment: @naught101: Thanks for the wording change -- I think your suggested doc change will be a good one. I've added a comment which you can access via the "review" link to the right of the link for your mp.map.starmap.p

[issue28668] instanciation of multiprocessing.Queue raises ImportError in test_logging

2016-11-11 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue28668> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue26924] android: test_concurrent_futures fails

2016-11-11 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue26924> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue28696] imap from ThreadPool hangs by an exception in a generator function

2016-11-15 Thread Davin Potts
Davin Potts added the comment: Using the supplied example, I was unable to reproduce what you described using either 3.5 or 3.6-beta on OS X 10.11. What platform are you using? (Perhaps it is platform specific.) -- nosy: +davin ___ Python tracker

[issue28696] imap from ThreadPool hangs by an exception in a generator function

2016-11-15 Thread Davin Potts
Davin Potts added the comment: If it only occurs in 3.4.x, is moving to 3.5 an option for you? -- versions: -Python 3.5 ___ Python tracker <http://bugs.python.org/issue28

[issue28699] Imap from ThreadPool behaves unexpectedly

2016-11-16 Thread Davin Potts
Davin Potts added the comment: To tie in the example given by @elias in issue28625, this inconsistency in behavior is not limited to ThreadPool -- it appears with a process Pool as well: from multiprocessing import Pool def double(x): return 2 * x def get_numbers(): raise Exception

[issue28625] multiprocessing.Pool.imap swallows exceptions thrown by generators

2016-11-16 Thread Davin Potts
Davin Potts added the comment: In issue28699, an important clue has been identified (a variation on the generator triggers a different, inconsistent behavior). I will merge this issue into that one to keep us all on the same page. -- superseder: -> Imap from ThreadPool beha

[issue28699] Imap from ThreadPool behaves unexpectedly

2016-11-16 Thread Davin Potts
Davin Potts added the comment: This inconsistent behavior in imap on both Pool and ThreadPool is not what I would expect. -- versions: +Python 3.6, Python 3.7 ___ Python tracker <http://bugs.python.org/issue28

[issue28696] imap from ThreadPool hangs by an exception in a generator function

2016-11-16 Thread Davin Potts
Changes by Davin Potts : -- superseder: -> Imap from ThreadPool behaves unexpectedly ___ Python tracker <http://bugs.python.org/issue28696> ___ ___ Python-

[issue28699] Imap from ThreadPool behaves unexpectedly

2016-11-16 Thread Davin Potts
Davin Potts added the comment: Though it still lacks a proper test, I'm attaching a preliminary patch to address the problematic behavior in 3.5/3.6/default in the hopes that others might help test it. -- keywords: +patch Added file: http://bugs.python.org/file

[issue28699] Imap from ThreadPool behaves unexpectedly

2016-11-17 Thread Davin Potts
Davin Potts added the comment: @xiang.zhang: Your patch looks to be introducing a number of changes to the structure of the data being passed around between threads and even monitored/indirectly shared across processes. It's looking increasingly high risk to me. We already have logi

<    1   2   3   4