[issue10037] multiprocessing.pool processes started by worker handler stops working

Changes by Richard Oudkerk : -- resolution: -> later stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___

[issue10037] multiprocessing.pool processes started by worker handler stops working

Richard Oudkerk added the comment: > Ah, a working 'fork server' would be just as good. Only problem is that it depends on fd passing which is apparently broken on MacOSX. > Btw, Billiard now supports running Pool without threads, using > epoll/kqueue/select instead. So Celery uses that when

[issue10037] multiprocessing.pool processes started by worker handler stops working

Ask Solem added the comment: Later works, or just close it. I can open up a new issue to merge the improvements in billiard later. > The execv stuff certainly won't go in by Py3.3. There has not been > consensus that adding it is a good idea. > (I also have the unit tests passing with a "f

[issue10037] multiprocessing.pool processes started by worker handler stops working

Richard Oudkerk added the comment: > I think this issue can be closed, the worker handler is simply borked and > we could open up a new issue deciding how to fix it (merging billiard.Pool > or someting else). OK. I am not sure which option under "Resolution" should be chosen. "Later"? > (

[issue10037] multiprocessing.pool processes started by worker handler stops working

Ask Solem added the comment: Well, I still don't know exactly why restarting the socket read made it work, but the patch solved an issue where newly started pool processes would be stuck in socket read forever (happening to maybe 1/500 new processes) This and a dozen other pool related fixes

[issue10037] multiprocessing.pool processes started by worker handler stops working

Richard Oudkerk added the comment: It is not clear to me how to reproduce the bug. When you say "letting the workers terminate themselves" do mean calling sys.exit() or os._exit() in the submitted task? Are you trying to get the result of a task which caused the worker to exit? I'm not sure

[issue10037] multiprocessing.pool processes started by worker handler stops working

Changes by Richard Oudkerk : -- nosy: +sbt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue10037] multiprocessing.pool processes started by worker handler stops working

Sean Reifschneider added the comment: The attached patch does change the semantics somewhat, but I don't fully understand how much. In particular: It changes the "get()" call to be turned into "get(timeout=1.0)" if inqueue doesn't have a _reader attribute. In the case that inqueue doesn't ha

[issue10037] multiprocessing.pool processes started by worker handler stops working

Changes by Terry J. Reedy : -- versions: -Python 3.1 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://ma

[issue10037] multiprocessing.pool processes started by worker handler stops working

Changes by Nir Aides : -- nosy: +nirai ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/m

[issue10037] multiprocessing.pool processes started by worker handler stops working

Ray.Allen added the comment: Could you give an example code which can reproduce this issue? -- nosy: +ysj.ray ___ Python tracker ___

[issue10037] multiprocessing.pool processes started by worker handler stops working

New submission from Ask Solem : While working on an "autoscaling" (yes, people call it that...) feature for Celery, I noticed that the processes created by the _handle_workers thread doesn't always work. I have reproduced this in general, by just using the maxtasksperchild feature and letting