[issue34781] infinite waiting in multiprocessing.Pool

2018-09-26 Thread Tomáš Bouda
Tomáš Bouda added the comment: By now I have spent several days trying to reproduce the behaviour in production environment with debugger attached. Unfortunately, no success. On the other hand yesterday the application froze, again, and colleague today experienced the problem in his script

[issue34781] infinite waiting in multiprocessing.Pool

2018-09-25 Thread Tomáš Bouda
Tomáš Bouda added the comment: Oh, I should add that by decreasing number of workers to 4 or 8 the problem disappeared, at least to the extent when I wasn't able to reproduce it on any environment. -- ___ Python tracker <https://bugs.py

[issue34781] infinite waiting in multiprocessing.Pool

2018-09-25 Thread Tomáš Bouda
Tomáš Bouda added the comment: It's very difficult to reproduce. In this example to get stuck on 3.6/OSX I need to attach debugger. However, the freeze happens regardless of print/logging, even def f(): pass can get stuck. os.write() made no difference and frozen, as well, as I've

[issue34781] infinite waiting in multiprocessing.Pool

2018-09-23 Thread Tomáš Bouda
Tomáš Bouda added the comment: After more digging, I found that the following happens: popen_fork.py -> _launch(self, process_obj) -> self.pid = os.fork() When I let process (both child and parent) print resulting pid, on freezing I can see: a) 50-times pid > 0 b) 49-times pid =

[issue34781] infinite waiting in multiprocessing.Pool

2018-09-23 Thread Tomáš Bouda
New submission from Tomáš Bouda : I have encountered a possible bug inside multiprocessing.Pool which behaves like race-condition while I don't believe it is a typical one. Simply put, Pool from time to time freezes. It is occasional and hard to reproduce, but e.g. unit-tests running