Re: [Python-Dev] GIL behaviour under Windows

2009-10-22 Thread Sturla Molden
Antoine Pitrou skrev: This number lacks the elapsed time. 61 switches in one second is probably enough, the same amount of switches in 10 or 20 seconds is too small (at least for threads needing good responsivity, e.g. I/O threads). Also, "fair" has to take into account the average latency and i

Re: [Python-Dev] GIL behaviour under Windows

2009-10-22 Thread Phillip Sitbon
I know I already posted some relevant threads to the other discussion, but I wanted to point out a couple of specific comments on GIL fairness from the discussion: http://mail.python.org/pipermail/python-dev/2009-May/089752.html http://mail.python.org/pipermail/python-dev/2009-May/089755.html - P

Re: [Python-Dev] GIL behaviour under Windows

2009-10-22 Thread Antoine Pitrou
Tres Seaver palladion.com> writes: > > I read Sturla as saying there were 99,939 switches out of a possible > 100,000, with sys.checkinterval set to 100. Oops, you're right. But depending on the elapsed time (again :-)), it may be too high, because too many switches per second will add a lot of

Re: [Python-Dev] GIL behaviour under Windows

2009-10-22 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou wrote: > Sturla Molden molden.no> writes: >> With two threads and a check interval og 100, only 61 of 10 check >> intervals failed to produce a thread-switch in the interpreter. I'd call >> that rather fair. > > This number lack

Re: [Python-Dev] GIL behaviour under Windows

2009-10-22 Thread Antoine Pitrou
Sturla Molden molden.no> writes: > > With two threads and a check interval og 100, only 61 of 10 check > intervals failed to produce a thread-switch in the interpreter. I'd call > that rather fair. This number lacks the elapsed time. 61 switches in one second is probably enough, the same

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Sturla Molden
Sturla Molden skrev: However, David Beazley is not talking about Windows. Since the GIL is apparently not a mutex on Windows, it could behave differently. So I wrote a small script that contructs a GIL battle, and record how often a check-interval results in a thread-switch or not. For monito

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Sturla Molden
Antoine Pitrou skrev: Kristján sent me a patch which I applied and is supposed to fix this. Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero latency with the Pi calculation in the background) than under Linux, although it may be caused by the limited resolution of time

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou wrote: > Le mercredi 21 octobre 2009 à 12:42 -0500, John Arbash Meinel a écrit : >> You can use time.clock() instead to get <15ms resolution. Changing all >> instances of 'time.time' to 'time.clock' gives me this result: > [snip] >> ---

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Antoine Pitrou
Le mercredi 21 octobre 2009 à 12:42 -0500, John Arbash Meinel a écrit : > > You can use time.clock() instead to get <15ms resolution. Changing all > instances of 'time.time' to 'time.clock' gives me this result: [snip] > > --- Latency --- > > Background CPU task: Pi calculation (Python) > > CPU

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread John Arbash Meinel
Antoine Pitrou wrote: > Sturla Molden molden.no> writes: >> It does not crash the interpreter, but it seems it can deadlock. > > Kristján sent me a patch which I applied and is supposed to fix this. > Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero > latency with the

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Antoine Pitrou
Sturla Molden molden.no> writes: > > It does not crash the interpreter, but it seems it can deadlock. Kristján sent me a patch which I applied and is supposed to fix this. Anyway, thanks for the numbers. The GIL does seem to fare a bit better (zero latency with the Pi calculation in the backgro

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Sturla Molden
Sturla Molden skrev: does not crash the interpreter, but it seems it can deadlock. Here is what I get con a quadcore (Vista, Python 2.6.3). This what I get with affinity set to CPU 3. There are deadlocks happening at random locations in ccbench.py. It gets worse with affinity set to one pr

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Sturla Molden
Antoine Pitrou skrev: (*) http://svn.python.org/view/sandbox/trunk/ccbench/ I´ve run it twice on my dual core machine. It hangs every time, but not in the same place: D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py Ah, you should report a bug then. ccbench is pure Python

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread John Arbash Meinel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Antoine Pitrou wrote: >> I don't really know how this test works, so I won't claim to understand >> the results either. However, here you go: > > Thanks. > > Interesting results. I wonder what they would be like on a multi-core > machine. The GIL see

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Antoine Pitrou
> > (*) http://svn.python.org/view/sandbox/trunk/ccbench/ > > I´ve run it twice on my dual core machine. It hangs every time, but not in > the same place: > D:\pydev\python\trunk\PCbuild>python.exe \tmp\ccbench.py Ah, you should report a bug then. ccbench is pure Python (and not particularly e

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Kristján Valur Jónsson
> -Original Message- > Could you try ccbench (*) under Windows? The only Windows system I have > here is > a qemu virtual machine and it wouldn't be very realistic to do > concurrency > measurements on it. > > (*) http://svn.python.org/view/sandbox/trunk/ccbench/ I´ve run it twice on my

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Kristján Valur Jónsson
> -Original Message- > From: python-dev-bounces+kristjan=ccpgames@python.org > [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf > Of Antoine Pitrou > Sent: 21. október 2009 10:52 > To: python-dev@python.org > Subject: Re: [Python-Dev] GIL be

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Scott Dial
Antoine Pitrou wrote: > Interesting results. I wonder what they would be like on a multi-core > machine. The GIL seems to behave perfectly on your setup (no performance > degradation due to concurrency, and zero latencies). You are correct, my machine is a single-core system. I don't have any mult

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Antoine Pitrou
> I don't really know how this test works, so I won't claim to understand > the results either. However, here you go: Thanks. Interesting results. I wonder what they would be like on a multi-core machine. The GIL seems to behave perfectly on your setup (no performance degradation due to concurren

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Scott Dial
Antoine Pitrou wrote: > Could you try ccbench (*) under Windows? The only Windows system I have here > is > a qemu virtual machine and it wouldn't be very realistic to do concurrency > measurements on it. > > (*) http://svn.python.org/view/sandbox/trunk/ccbench/ > I don't really know how this t

Re: [Python-Dev] GIL behaviour under Windows

2009-10-21 Thread Antoine Pitrou
Hello Kristjan, > This depends entirely on the platform and primitives used to implement the > GIL. > I'm interested in windows. Could you try ccbench (*) under Windows? The only Windows system I have here is a qemu virtual machine and it wouldn't be very realistic to do concurrency measurement