Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Nathaniel Smith
On Thu, May 25, 2017 at 12:01 PM, Eric Snow wrote: > More significantly, I genuinely believe that isolated > interpreters in the same process is a tool that many people will find > extremely useful and will help the Python community. Consequently, > exposing subinterpreters in the stdlib would re

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Ronald Oussoren
> On 25 May 2017, at 19:03, Eric Snow wrote: > > On Wed, May 24, 2017 at 8:30 PM, Guido van Rossum wrote: >> Hm... Curiously, I've heard a few people at PyCon > > I'd love to get in touch with them and discuss the situation. I've > spoken with Graham Dumpleton on several occasions about > sub

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Petr Viktorin
On 05/25/2017 09:01 PM, Eric Snow wrote: On Thu, May 25, 2017 at 11:19 AM, Nathaniel Smith wrote: My impression is that the code to support them inside CPython is fine, but they're broken and not very useful in the sense that lots of C extensions don't really support them, so in practice you ca

Re: [Python-ideas] tweaking the file system path protocol

2017-05-26 Thread Koos Zevenhoven
On Wed, May 24, 2017 at 3:41 AM, Steven D'Aprano wrote: > On Wed, May 24, 2017 at 12:18:16AM +0300, Serhiy Storchaka wrote: >> I don't know a reasonable use case for this feature. The __fspath__ >> method of str or bytes subclasses returning something not equivalent to >> self looks confusing to m

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Stephan Houben
Hi all, Personally I feel that the current subinterpreter support falls short in the sense that it still requires a single GIL across interpreters. If interpreters would have their own individual GIL, we could have true shared-nothing multi-threaded support similar to Javascript's "Web Workers".

Re: [Python-ideas] tweaking the file system path protocol

2017-05-26 Thread Koos Zevenhoven
On Wed, May 24, 2017 at 5:52 PM, Wolfgang Maier wrote: > On 05/24/2017 02:41 AM, Steven D'Aprano wrote: >> >> >> It would be annoying and inconsistent if int(x) avoided calling __int__ >> on int subclasses. But that's exactly what happens with fspath and str. >> I see that as a bug, not a feature:

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Nick Coghlan
On 26 May 2017 at 22:08, Stephan Houben wrote: > Hi all, > > Personally I feel that the current subinterpreter support falls short > in the sense that it still requires > a single GIL across interpreters. > > If interpreters would have their own individual GIL, > we could have true shared-nothing

Re: [Python-ideas] tweaking the file system path protocol

2017-05-26 Thread Koos Zevenhoven
Accidentally sent the email before it was done. Additions / corrections below: On Fri, May 26, 2017 at 3:58 PM, Koos Zevenhoven wrote: > On Wed, May 24, 2017 at 5:52 PM, Wolfgang Maier >> >> - makes the "path.__fspath__() if hasattr(path, "__fspath__") else path" >> idiom consistent for subclasse

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Stephan Houben
Hi Nick, As far as I understand, the (to me) essential difference between your approach and my proposal is that: Approach 1 (PEP-489): * Single (global) GIL. * PyObject's may be shared across interpreters (zero-copy transfer) Approach 2 (mine) * Per-interpreter GIL. * PyObject's must

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Nick Coghlan
On 26 May 2017 at 23:49, Stephan Houben wrote: > Hi Nick, > > As far as I understand, the (to me) essential difference between your > approach and my proposal is that: > > Approach 1 (PEP-489): >* Single (global) GIL. >* PyObject's may be shared across interpreters (zero-copy transfer) > >

Re: [Python-ideas] Exposing CPython's subinterpreter C-API in the stdlib.

2017-05-26 Thread Guido van Rossum
On Fri, May 26, 2017 at 8:28 AM, Nick Coghlan wrote: > [...] assuming the rest of idea works out > well, we'd eventually like to move to a tiered model where the GIL > becomes a read/write lock. Most code execution in subinterpreters > would then only need a read lock on the GIL, and hence could

[Python-ideas] a memory-efficient variant of itertools.tee

2017-05-26 Thread Stephan Houben
Hi all, The itertools.tee function can hold on to objects "unnecessarily". In particular, if you do iter2 = itertools.tee(iter1, 2)[0] i.e. you "leak" one of the returned iterators, then all returned objects are not collected until also iter2 is collected. I propose a different implementation,