Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-04 Thread Josiah Carlson
On Tue, Nov 4, 2008 at 5:31 AM, Leonardo Santagada <[EMAIL PROTECTED]> wrote: > > On Nov 4, 2008, at 12:11 AM, Josiah Carlson wrote: > >> There is no shortage of algorithms (such as matrix multiplication) that >> are parallelizable but not particularly good candidates for an IPC-based >> multiproce

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-04 Thread Leonardo Santagada
On Nov 4, 2008, at 12:11 AM, Josiah Carlson wrote: There is no shortage of algorithms (such as matrix multiplication) that are parallelizable but not particularly good candidates for an IPC-based multiprocessing paradigm. Ahh, but those algorithms aren't going to be written in Python; they

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-03 Thread Josiah Carlson
On Mon, Nov 3, 2008 at 10:59 AM, Curt Hagenlocher <[EMAIL PROTECTED]>wrote: > On Mon, Nov 3, 2008 at 11:50 AM, Josiah Carlson <[EMAIL PROTECTED]>wrote: > >> On Sun, Nov 2, 2008 at 3:51 PM, <[EMAIL PROTECTED]> wrote: >> >>> >>>Antoine> I think it is important to remind that the GIL doesn't >>>

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-03 Thread Matthieu Brucher
2008/11/3 <[EMAIL PROTECTED]>: > >>> I believe everyone here knows that. I believe what most people are >>> clamoring for is to make "full use of their multi-CPU resources in a >>> single process". > >Josiah> Which is, arguably, silly. As we've seen in the last 2 months >Josi

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-03 Thread skip
>> I believe everyone here knows that. I believe what most people are >> clamoring for is to make "full use of their multi-CPU resources in a >> single process". Josiah> Which is, arguably, silly. As we've seen in the last 2 months Josiah> with Chrome, multiple processes for

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-03 Thread Curt Hagenlocher
On Mon, Nov 3, 2008 at 11:50 AM, Josiah Carlson <[EMAIL PROTECTED]>wrote: > On Sun, Nov 2, 2008 at 3:51 PM, <[EMAIL PROTECTED]> wrote: > >> >>Antoine> I think it is important to remind that the GIL doesn't prevent >>Antoine> (almost) true multithreading. The only thing it prevents is >>

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-03 Thread Josiah Carlson
On Sun, Nov 2, 2008 at 3:51 PM, <[EMAIL PROTECTED]> wrote: > >Antoine> I think it is important to remind that the GIL doesn't prevent >Antoine> (almost) true multithreading. The only thing it prevents is >Antoine> full use of multi-CPU resources in a single process. > > I believe every

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-03 Thread Adam Olsen
On Sun, Nov 2, 2008 at 4:33 PM, Greg Ewing <[EMAIL PROTECTED]> wrote: > Eric Smith wrote: > >> I'd gladly trade deterministic destruction (due to reference counting or >> any other mechanism) for improved performance. > > Another thing to consider is that refcounting spreads out the > time spent do

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Eric Smith
[EMAIL PROTECTED] wrote: Eric> I consider it a bug to rely on reference counting to close files, We can mostly have our cake and eat it too using the "with" statement. In most cases it should be sufficient I would think. True, and I meant to mention that. But unfortunately, my work projec

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread skip
Eric> I consider it a bug to rely on reference counting to close files, We can mostly have our cake and eat it too using the "with" statement. In most cases it should be sufficient I would think. Skip ___ Python-Dev mailing list Python-Dev@python

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread skip
Antoine> I think it is important to remind that the GIL doesn't prevent Antoine> (almost) true multithreading. The only thing it prevents is Antoine> full use of multi-CPU resources in a single process. I believe everyone here knows that. I believe what most people are clamoring for

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Greg Ewing
Eric Smith wrote: I'd gladly trade deterministic destruction (due to reference counting or any other mechanism) for improved performance. Another thing to consider is that refcounting spreads out the time spent doing GC evenly over the execution of the program, so that you don't get pauses occ

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Adam Olsen
On Sun, Nov 2, 2008 at 2:34 PM, Eric Smith <[EMAIL PROTECTED]> wrote: > Giovanni Bajo wrote: >> >> [[ my 0.2: it would be a great loss if we lose reference-counting semantic >> (eg: objects deallocated as soon as they exit the scope). I would bargain >> that for a noticable speed increase of course

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Eric Smith
Giovanni Bajo wrote: [[ my 0.2: it would be a great loss if we lose reference-counting semantic (eg: objects deallocated as soon as they exit the scope). I would bargain that for a noticable speed increase of course, but my own experience with standard GCs from other languages has been less tha

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Stefan Behnel
Michael Foord wrote: > Moving more C extensions to an implementation based on ctypes would be > enormously useful for PyPy, IronPython and Jython, but ctypes is not yet > as portable as Python itself which could be an issue (although one worth > resolving). In the same line, moving more extensions

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Antoine Pitrou
Hi, Jesse Noller gmail.com> writes: > If python were to have free threading, courtesy of a lack > of the GIL, it would help those people quite a bit. Sometimes you just > need shared state. Myself? I used multiprocess *and* threads all the > time for various reasons. I think it is important to

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-02 Thread Giovanni Bajo
On Sun, 02 Nov 2008 10:21:26 +1000, Nick Coghlan wrote: > Maciej Fijalkowski wrote: >>> ... >> >>> We know it is the plan for PyPy to work in this way, and also that >>> Jython and Ironpython works like that (using the host vm's GC), so it >>> seems to be somehow agreeable with the python semanti

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-01 Thread Nick Coghlan
Maciej Fijalkowski wrote: >> ... > >> We know it is the plan for PyPy to work in this way, and also that >> Jython and Ironpython works like that (using the host vm's GC), so it >> seems to be somehow agreeable with the python semantics (perhaps not >> really with __del__ but they are not really n

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-11-01 Thread Maciej Fijalkowski
> ... > > We know it is the plan for PyPy to work in this way, and also that > Jython and Ironpython works like that (using the host vm's GC), so it > seems to be somehow agreeable with the python semantics (perhaps not > really with __del__ but they are not really nice anyway). > PyPy has a semi

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Adam Olsen
On Fri, Oct 31, 2008 at 11:55 AM, Brian Granger <[EMAIL PROTECTED]> wrote: >>> Has anyone made the argument for keeping the GIL to discourage >>> threading? >> >> Oooh, you are on to my secret plan! :-) > > I completely agree that there are other approaches to parallelism and > concurrency that are

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Brian Granger
>> Has anyone made the argument for keeping the GIL to discourage >> threading? > > Oooh, you are on to my secret plan! :-) I completely agree that there are other approaches to parallelism and concurrency that are much better than threading. However, I don't think this is a good argument for hav

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Guido van Rossum
On Fri, Oct 31, 2008 at 4:59 AM, Calvin Spealman <[EMAIL PROTECTED]> wrote: > Has anyone made the argument for keeping the GIL to discourage > threading? Oooh, you are on to my secret plan! :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) _

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Thomas Heller
Victor Stinner schrieb: > Le Friday 31 October 2008 14:13:01 Christian Heimes, vous avez écrit : >> ctypes is also missing utilities to write code that works on 32 and >> 64bit platforms. Without a tool like >> http://pypi.python.org/pypi/ctypes_configure it's very hard and tedious >> to avoid and

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Thomas Heller
Christian Heimes schrieb: > Nick Coghlan wrote: >> If I recall correctly, the main blocker to ctypes portability is libffi >> portability. So if anyone would like to see ctypes on more platforms, >> then that's the limitation they really need to attack. > > ctypes is also missing utilities to writ

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread skip
Calvin> Has anyone made the argument for keeping the GIL to discourage Calvin> threading? I'm only throwing this out there and I'm sure we'd Calvin> want to improve things no matter what, but I would like to voice Calvin> the concern anyway. We all know there are people who think

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Jesse Noller
On Fri, Oct 31, 2008 at 9:17 AM, Michael Foord <[EMAIL PROTECTED]> wrote: > Calvin Spealman wrote: >> >> Has anyone made the argument for keeping the GIL to discourage >> threading? I'm only throwing this out there and I'm sure we'd want to >> improve things no matter what, but I would like to voic

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Victor Stinner
Le Friday 31 October 2008 14:13:01 Christian Heimes, vous avez écrit : > ctypes is also missing utilities to write code that works on 32 and > 64bit platforms. Without a tool like > http://pypi.python.org/pypi/ctypes_configure it's very hard and tedious > to avoid and fix segfaults. I wrote some c

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Michael Foord
Calvin Spealman wrote: Has anyone made the argument for keeping the GIL to discourage threading? I'm only throwing this out there and I'm sure we'd want to improve things no matter what, but I would like to voice the concern anyway. We all know there are people who think threading is the answer t

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Christian Heimes
Nick Coghlan wrote: If I recall correctly, the main blocker to ctypes portability is libffi portability. So if anyone would like to see ctypes on more platforms, then that's the limitation they really need to attack. ctypes is also missing utilities to write code that works on 32 and 64bit pla

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Oleg Broytmann
On Fri, Oct 31, 2008 at 07:59:01AM -0400, Calvin Spealman wrote: > Has anyone made the argument for keeping the GIL to discourage > threading? I haven't, but I would support such argument. In my humble opinion multithreading is rarely a good answer, and for multicore CPUs it's a wrong answer. F

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Calvin Spealman
Has anyone made the argument for keeping the GIL to discourage threading? I'm only throwing this out there and I'm sure we'd want to improve things no matter what, but I would like to voice the concern anyway. We all know there are people who think threading is the answer to all things, and who don

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Adam Olsen
On Fri, Oct 31, 2008 at 2:11 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Adam Olsen wrote: >> On Fri, Oct 31, 2008 at 12:24 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: >>> Neil Schemenauer wrote: Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: > For a student project in a course on vi

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Nick Coghlan
Adam Olsen wrote: > On Fri, Oct 31, 2008 at 12:24 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: >> Neil Schemenauer wrote: >>> Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: For a student project in a course on virtual machines, we are evaluating the possibility to experiment with remov

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-31 Thread Adam Olsen
On Fri, Oct 31, 2008 at 12:24 AM, Nick Coghlan <[EMAIL PROTECTED]> wrote: > Neil Schemenauer wrote: >> Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: >>> For a student project in a course on virtual machines, we are >>> evaluating the possibility to experiment with removing the GIL >>> from CPy

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Nick Coghlan
Michael Foord wrote: > Moving more C extensions to an implementation based on ctypes would be > enormously useful for PyPy, IronPython and Jython, but ctypes is not yet > as portable as Python itself which could be an issue (although one worth > resolving). If I recall correctly, the main blocker

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Nick Coghlan
Neil Schemenauer wrote: > Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: >> For a student project in a course on virtual machines, we are >> evaluating the possibility to experiment with removing the GIL >> from CPython > > Hi, > > It's great to hear of this kind of project. I think what you

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Neil Schemenauer
Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: > For a student project in a course on virtual machines, we are > evaluating the possibility to experiment with removing the GIL > from CPython Hi, It's great to hear of this kind of project. I think what you want to do is difficult but possible

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread James Y Knight
On Oct 30, 2008, at 1:31 PM, Eric Smith wrote: Guido van Rossum wrote: No offense taken. The V8 experience makes me feel much more optimistic that they might actually pull this off. (I'm still skeptical about support for extension modules, withougt which CPython is pretty lame.) The need

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Michael Foord
Guido van Rossum wrote: On Thu, Oct 30, 2008 at 10:31 AM, Eric Smith <[EMAIL PROTECTED]> wrote: Guido van Rossum wrote: No offense taken. The V8 experience makes me feel much more optimistic that they might actually pull this off. (I'm still skeptical about support for extension modules

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Guido van Rossum
On Thu, Oct 30, 2008 at 10:31 AM, Eric Smith <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: >> >> No offense taken. The V8 experience makes me feel much more optimistic >> that they might actually pull this off. (I'm still skeptical about >> support for extension modules, withougt which CPyth

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Eric Smith
Guido van Rossum wrote: No offense taken. The V8 experience makes me feel much more optimistic that they might actually pull this off. (I'm still skeptical about support for extension modules, withougt which CPython is pretty lame.) The need to modify all extension modules is the usual non-star

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Guido van Rossum
On Thu, Oct 30, 2008 at 10:20 AM, VanL <[EMAIL PROTECTED]> wrote: > VanL wrote: > >> Just an FYI, > > ...and the FYI was to no one in particular, just interesting information > from the PyPy list. It is just unfortunate timing that made it look like > a cheeky reply to Guido. No offense taken. The

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Leif Walsh
On Thu, Oct 30, 2008 at 1:07 PM, VanL <[EMAIL PROTECTED]> wrote: > Just an FYI, these two particular students already introduced themselves > on the PyPy list. Paolo is a masters student with experience in the > Linux kernel; Sigurd is a PhD candidate. > > Their professor is Lars Bak, the lead arch

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread VanL
VanL wrote: > Just an FYI, ...and the FYI was to no one in particular, just interesting information from the PyPy list. It is just unfortunate timing that made it look like a cheeky reply to Guido. ___ Python-Dev mailing list Python-Dev@python.org htt

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread VanL
Sigurd Torkel Meldgaard wrote: > Hi to all Python developers > > For a student project in a course on virtual machines, we are > evaluating the possibility to > experiment with removing the GIL from CPython... Just an FYI, these two particular students already introduced themselves on the PyPy li

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Guido van Rossum
It's not that I have any love for the GIL, it just is the best compromise I could find. I expect that you won't be able to do better, but I wish you luck anyway. On Thu, Oct 30, 2008 at 9:13 AM, Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: > Hi to all Python developers > > For a student proj

Re: [Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Jesse Noller
On Thu, Oct 30, 2008 at 12:13 PM, Sigurd Torkel Meldgaard <[EMAIL PROTECTED]> wrote: > Hi to all Python developers > > For a student project in a course on virtual machines, we are > evaluating the possibility to > experiment with removing the GIL from CPython > > We have read the arguments against

[Python-Dev] Fwd: Removal of GIL through refcounting removal.

2008-10-30 Thread Sigurd Torkel Meldgaard
Hi to all Python developers For a student project in a course on virtual machines, we are evaluating the possibility to experiment with removing the GIL from CPython We have read the arguments against doing this at http://www.python.org/doc/faq/library/#can-t-we-get-rid-of-the-global-interpreter-