On Wed, Sep 9, 2009 at 23:56, Nick Coghlan wrote:
> Thomas Wouters wrote:
> > Your idea of making this an API called a 'fork lock' or something
> > sounds good (though I think it needs a better name. PyBeforeFork &
> > PyAfterFork?). The subprocess module, for example, disables garb
On Fri, Sep 11, 2009 at 08:06, Cameron Simpson wrote:
> On 25Jul2009 10:25, Gregory P. Smith wrote:
> | On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters
> wrote:
> | > So attached (and at http://codereview.appspot.com/96125/show ) is a
> | > preliminary fix, correcting the problem with os.fork(),
On 25Jul2009 10:25, Gregory P. Smith wrote:
| On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote:
| > So attached (and at http://codereview.appspot.com/96125/show ) is a
| > preliminary fix, correcting the problem with os.fork(), os.forkpty() and
| > os.fork1(). This doesn't expose a general AP
Thomas Wouters wrote:
> Your idea of making this an API called a 'fork lock' or something
> sounds good (though I think it needs a better name. PyBeforeFork &
> PyAfterFork?). The subprocess module, for example, disables garbage
> collection before forking and restores it afterwar
On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith wrote:
> On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters wrote:
> >
> >
> > On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith wrote:
> >>
> >> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters
> wrote:
> >> >
> >> > So attached (and at http://codereview.
On Wed, Sep 9, 2009 at 11:36 AM, Thomas Wouters wrote:
>
> On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith wrote:
>>
>> Take a look at http://code.google.com/p/python-atfork/ which I created
>> to address the general fork+threading with locks held causing
>> deadlocks issue with many standard libra
On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters wrote:
>
>
> On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith wrote:
>>
>> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote:
>> >
>> > So attached (and at http://codereview.appspot.com/96125/show ) is a
>> > preliminary fix, correcting the problem
On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith wrote:
> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote:
> >
> > So attached (and at http://codereview.appspot.com/96125/show ) is a
> > preliminary fix, correcting the problem with os.fork(), os.forkpty() and
> > os.fork1(). This doesn't exp
On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote:
>
> So attached (and at http://codereview.appspot.com/96125/show ) is a
> preliminary fix, correcting the problem with os.fork(), os.forkpty() and
> os.fork1(). This doesn't expose a general API for C code to use, for two
> reasons: it's not ea
So attached (and at http://codereview.appspot.com/96125/show ) is a
preliminary fix, correcting the problem with os.fork(), os.forkpty() and
os.fork1(). This doesn't expose a general API for C code to use, for two
reasons: it's not easy, and I need this fix more than I need the API change
:-) (I ac
On Mon, Jul 20, 2009 at 11:26, Mike Klaas wrote:
>
>
> On Thu, Jul 16, 2009 at 1:08 PM, Thomas Wouters wrote:
>
>>
>> Picking up a rather old discussion... We encountered this bug at Google
>> and I'm now "incentivized" to fix it.
>>
>> For a short recap: Python has an import lock that prevents
On Thu, Jul 16, 2009 at 1:08 PM, Thomas Wouters wrote:
>
> Picking up a rather old discussion... We encountered this bug at Google and
> I'm now "incentivized" to fix it.
>
> For a short recap: Python has an import lock that prevents more than one
> thread from doing an import at any given time.
On Thu, May 18, 2006 at 3:02 PM, "Martin v. Löwis" wrote:
>
> Right. With some googling, I found that one solution is pthread_atexit:
> a pthread_atexit handler is a triple (before, in_parent, in_child).
> You set it to (acquire, release, release).
Did you mean pthread_atfork() ?
> When somebody
Thomas Wouters wrote:
> Picking up a rather old discussion... We encountered this bug at Google
> and I'm now "incentivized" to fix it.
>
> For a short recap: Python has an import lock that prevents more than one
> thread from doing an import at any given time. However, unlike most of
> the locks
On Thu, May 18, 2006 at 20:02, "Martin v. Löwis" wrote:
> Nick Coghlan wrote:
> > And if I understand it correctly, it falls under the category that
> > waiting for another thread while holding the import lock is a *really*
> > bad idea from a thread safety point of view.
> >
> > The thing with t
Ronald Oussoren wrote:
> Wouldn't this specific problem be fixed if os.fork were to acquire the
> import lock before calling fork(2)?
Right. Of course, you need to release the lock then both in the parent
and the child. Also, all places that currently do PyOs_AfterFork should
get modified (i.e. fo
Nick Coghlan wrote:
> And if I understand it correctly, it falls under the category that
> waiting for another thread while holding the import lock is a *really*
> bad idea from a thread safety point of view.
>
> The thing with the import-after-fork deadlock is that you can trigger it
> without ev
On 16-mei-2006, at 23:20, Martin v. Löwis wrote:
Nick Coghlan wrote:
Broadening the ifdef to cover all posix systems rather than just AIX
might be the right thing to do.
As I said: I doubt it is the right thing to do. Instead, the lock
should get released in the child process if it is held a
Martin v. Löwis wrote:
> Neal Norwitz wrote:
>>> As I said: I doubt it is the right thing to do. Instead, the lock
>>> should get released in the child process if it is held at the
>>> point of the fork.
>>>
>> I'm not sure if this bug can be reproduced now, but the comment in
>> warnings.py points
Neal Norwitz wrote:
>> As I said: I doubt it is the right thing to do. Instead, the lock
>> should get released in the child process if it is held at the
>> point of the fork.
>>
> I'm not sure if this bug can be reproduced now, but the comment in
> warnings.py points to:
>
>http://python.org/
On 5/16/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Nick Coghlan wrote:
> > Broadening the ifdef to cover all posix systems rather than just AIX
> > might be the right thing to do.
>
> As I said: I doubt it is the right thing to do. Instead, the lock
> should get released in the child proces
On 5/16/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
Rotem should simply avoid to fork() in the toplevel code of a module.this is not what happened.we called subprocess which itself called fork.as a subprocess bug this could be easily fixed by moving the import in "os._execve".
we have made t
Nick Coghlan wrote:
> Broadening the ifdef to cover all posix systems rather than just AIX
> might be the right thing to do.
As I said: I doubt it is the right thing to do. Instead, the lock
should get released in the child process if it is held at the
point of the fork.
I agree that we should f
Yair Chuchem wrote:
> Rotem should simply avoid to fork() in the toplevel code of a module.
>
>
> this is not what happened.
> we called subprocess which itself called fork.
Ok - but are you calling subprocess in the context of code that is just
being imported? Or is the caller of the code t
Rotem Yaari wrote:
>> Rotem should simply avoid to fork() in the toplevel code of a module.
>>
>
> That's exactly the problem - the fork is not done at the module level,
> and the reason for the deadlock is imports inside functions performed
> by various library functions.
> This can happen virtua
Martin v. Löwis wrote:
> So if the the import lock was held during the fork call, the new thread
> will hold the import lock of the new process, and subsequent imports
> will block.
> However, then the problem is not with the execve implementation, but
> with the fact that the import lock was held
Nick Coghlan wrote:
> Rotem took the time to explain the problem to me more fully in private email,
> and it appears that using import statements inside functions is currently
> fundamentally unsafe in a posix environment where both multiple threads and
> fork() are used. If any thread other tha
Guido van Rossum wrote:
> Yeah, I think imports inside functions are overused.
Rotem took the time to explain the problem to me more fully in private email,
and it appears that using import statements inside functions is currently
fundamentally unsafe in a posix environment where both multiple t
Rotem Yaari wrote:
> This has occurred on Python 2.4.1 on a 2.4.27 Linux kernel.
>
> Preliminary analysis of the hang shows that the child process blocks
> upon entering the execvp function, in which the import_lock is acquired
> due to the following line:
>
> def _ execvpe(file, args, env=No
Yeah, I think imports inside functions are overused.
On 5/9/06, Rotem Yaari <[EMAIL PROTECTED]> wrote:
> Hello everyone!
>
> We have been encountering several deadlocks in a threaded Python
> application which calls subprocess.Popen (i.e. fork()) in some of its
> threads.
>
> This has occurred on
30 matches
Mail list logo
