Re: [Python-Dev] PEP 468

2016-06-10 Thread Eric Snow
On Fri, Jun 10, 2016 at 11:54 AM, Franklin? Lee wrote: > Eric, have you any work in progress on compact dicts? Nope. I presume you are talking the proposal Raymond made a while back. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

2016-06-10 Thread Eric Snow
On Fri, Jun 10, 2016 at 11:29 AM, Nick Coghlan wrote: > On 10 June 2016 at 09:42, Eric Snow wrote: >> On Thu, Jun 9, 2016 at 2:39 PM, Nick Coghlan wrote: >>> That restriction would be comparable to what we do with __slots__ today: >>> >>> >&g

[Python-Dev] PEP 520: Ordered Class Definition Namespace (round 3)

2016-06-11 Thread Eric Snow
have been clarification. Guido, at this point I believe the PEP is ready for pronouncement. * I've included the most recent copy of the text below. Thanks. -eric == PEP: 520 Title: Ordered Class Definition Namespace Version: $Revision$ Last-Modified: $Date$ Au

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace (round 3)

2016-06-11 Thread Eric Snow
On Sat, Jun 11, 2016 at 7:51 PM, Émanuel Barry wrote: >> From: Eric Snow >> 1. if ``__definition_order__`` is defined in the class body then it >> must be a ``tuple`` of identifiers or ``None``; any other value >> will result in ``TypeError`` > > Why n

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

2016-06-16 Thread Eric Snow
Thanks for raising these good points, Nikita. I'll make sure the PEP reflects this discussion. (inline responses below...) -eric On Tue, Jun 14, 2016 at 3:41 AM, Nikita Nemkin wrote: > Is there any rationale for rejecting alternatives like: > > 1. Adding standard metaclass with ordered namespa

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

2016-06-16 Thread Eric Snow
On Thu, Jun 16, 2016 at 5:11 AM, Nikita Nemkin wrote: > I'll reformulate my argument: > > Ordered class namespaces are a minority use case that's already covered > by existing language features (custom metaclasses) and doesn't warrant > the extension of the language (i.e. making OrderedDict a buil

Re: [Python-Dev] PEP 487: Simpler customization of class creation

2016-06-16 Thread Eric Snow
On Thu, Jun 16, 2016 at 12:56 PM, Martin Teichmann wrote: > I am looking forward to a lot of comments on this! I'd be glad to give feedback on this, probably later today or tomorrow. In particular, I'd like to help resolve the intersection with PEP 520. :) -eric

Re: [Python-Dev] PEP 487: Simpler customization of class creation

2016-06-16 Thread Eric Snow
On Thu, Jun 16, 2016 at 3:36 PM, Nick Coghlan wrote: > I don't think that's a side note, I think it's an important point (and > relates to one of Nikita's questions as well): we have the option of > carving out certain aspects of PEP 520 as CPython implementation > details. > > In particular, the

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

2016-06-20 Thread Eric Snow
On Fri, Jun 17, 2016 at 7:32 PM, Nick Coghlan wrote: > The discussion in the PEP 487 thread made me realise that I'd like to > see a discussion in PEP 520 regarding whether or not to define > __definition_order__ for builtin types initialised via PyType_Ready or > created via PyType_FromSpec in ad

Re: [Python-Dev] PEP 520: Ordered Class Definition Namespace

2016-06-20 Thread Eric Snow
On Mon, Jun 20, 2016 at 9:49 AM, Guido van Rossum wrote: > I agree it's better to define the order as computed at runtime. I don't > think there's much of a point to mandate that all builtin/extension types > reveal their order too -- I doubt there will be many uses for that -- but I > don't want

Re: [Python-Dev] Review of PEP 520: Ordered Class Definition Namespace

2016-06-20 Thread Eric Snow
On Mon, Jun 20, 2016 at 10:32 AM, Guido van Rossum wrote: > - I don't like the exception for dunder names. I can see that __module__, > __name__ etc. that occur in every class are distractions, but since you're > adding methods, you should also add methods with dunder names like > __init__ or __g

[Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 4)

2016-06-20 Thread Eric Snow
I've updated PEP 520 to reflect a clearer focus on the definition order and less emphasis on OrderedDict. -eric === PEP: 520 Title: Preserving Class Attribute Definition Order Version: $Revision$ Last-Modified: $Date$ Author: Eric Snow Status: Draft

Re: [Python-Dev] PEP 487: Simpler customization of class creation

2016-06-21 Thread Eric Snow
On Mon, Jun 20, 2016 at 12:31 PM, Nikita Nemkin wrote: > Right. Ordered by default is a very serious implementation constraint. > It's only superior in a sense that it completely subsumes/obsoletes > PEP 520. Just to be clear, PEP 520 is more than just OrderedDict-by-default. In fact, the key poi

Re: [Python-Dev] Review of PEP 520: Ordered Class Definition Namespace

2016-06-21 Thread Eric Snow
On Tue, Jun 21, 2016 at 3:21 PM, Nick Coghlan wrote: > It occurs to me that a settable __definition_order__ provides a > benefit that an ordered tp_dict doesn't: to get the "right" definition > order in something like Cython or dynamic type creation, you don't > need to carefully craft the order i

Re: [Python-Dev] PEP 487: Simpler customization of class creation

2016-06-21 Thread Eric Snow
On Tue, Jun 21, 2016 at 11:18 AM, Guido van Rossum wrote: > If we had had these semantics in the language from the start, there would have > been plenty uses of this order, and I suspect nobody would have considered > asking for __definition_order__. True. The key thing that __definition_order__

Re: [Python-Dev] PEP 487: Simpler customization of class creation

2016-06-21 Thread Eric Snow
On Tue, Jun 21, 2016 at 3:01 PM, Nick Coghlan wrote: > RIght, if *tp_dict itself* on type objects is guaranteed to be > order-preserviing, then we don't need to do anything except perhaps > provide a helper method or descriptor on type that automatically > filters out the dunder-attributes, and sp

Re: [Python-Dev] PEP XXX: Compact ordered dict

2016-06-23 Thread Eric Snow
On Mon, Jun 20, 2016 at 11:02 PM, INADA Naoki wrote: > On Tue, Jun 21, 2016 at 12:17 PM, Oleg Broytman wrote: >> (if a PEP is needed at all) > > I don't think so. My PEP is not for changing Python Language, > just describe implementation detail. > > Python 3.5 has new OrderedDict implemented in C

[Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

2016-06-24 Thread Eric Snow
- a clearer motivation section - include "dunder" names - 2 open questions (__slots__? drop read-only requirement?) -eric --- PEP: 520 Title: Preserving Class Attribute Definition Order Version: $Revision$ Last-Modified: $Date$ Author: Eric Snow Sta

Re: [Python-Dev] PEP 487: Simpler customization of class creation

2016-06-24 Thread Eric Snow
On Fri, Jun 24, 2016 at 1:50 PM, Nick Coghlan wrote: > Honestly though, I'm not sure this additional user-visible complexity > is worth it - "The default type metaclass has this new behaviour" is a > lot easier to document and explain than "We added a new opt-in > alternate metaclass that you can

Re: [Python-Dev] PEP XXX: Compact ordered dict

2016-06-24 Thread Eric Snow
There are a number of ways to make it work (mostly). However, I'll defer to Raymond on how strictly OrderedDict should "subclass" from dict. -eric On Thu, Jun 23, 2016 at 9:26 AM, INADA Naoki wrote: > On Fri, Jun 24, 2016 at 12:03 AM, Eric Snow > wrote: >> On Mo

Re: [Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

2016-06-24 Thread Eric Snow
On Fri, Jun 24, 2016 at 4:37 PM, Nick Coghlan wrote: > This version looks fine to me. \o/ > The definition order question has been dropped from PEP 487, so this > cross-reference doesn't really make sense any more :) Ah, so much for my appeal to authority. > I'd characterise this section at t

Re: [Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

2016-06-24 Thread Eric Snow
On Fri, Jun 24, 2016 at 5:56 PM, Random832 wrote: > On Fri, Jun 24, 2016, at 17:52, Eric Snow wrote: >> - 2 open questions (__slots__? drop read-only requirement?) > > It's worth noting that __slots__ itself doesn't have a read-only > requirement. It can be a tuple, a

Re: [Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

2016-06-28 Thread Eric Snow
On Sun, Jun 26, 2016 at 5:55 PM, Guido van Rossum wrote: >> On Fri, Jun 24, 2016 at 4:37 PM, Nick Coghlan wrote: >> > This version looks fine to me. > > Same to me, mostly. I've updated the PEP per everyone's comments [1], except I still haven't dropped the read-only __definition_order__ constra

Re: [Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

2016-06-28 Thread Eric Snow
On Tue, Jun 28, 2016 at 11:43 AM, Guido van Rossum wrote: > On Tue, Jun 28, 2016 at 10:30 AM, Eric Snow > wrote: >> I suppose I'm having a hard time letting go of the attractiveness of >> "read-only == complete". However, given that you've been pretty cle

Re: [Python-Dev] PEP 520: Preserving Class Attribute Definition Order (round 5)

2016-06-28 Thread Eric Snow
On Jun 28, 2016 2:56 PM, "Guido van Rossum" wrote: > > Awesome. That addresses my last concerns. PEP 520 is now accepted. > Congratulations! Yay! Thank you and to all those that gave such good feedback. -eric (phone) ___ Python-Dev mailing list Python

Re: [Python-Dev] Status of Python 3.6 PEPs?

2016-07-12 Thread Eric Snow
On Tue, Jul 12, 2016 at 3:26 AM, Victor Stinner wrote: > "PEP 520 -- Preserving Class Attribute Definition Order" > https://www.python.org/dev/peps/pep-0520/ > => accepted -- what is the status of its implementation? The implementation is currently under review (http://bugs.python.org/issue24254)

Re: [Python-Dev] PEP487: Simpler customization of class creation

2016-07-18 Thread Eric Snow
Great job, Martin! Thanks for seeing this through. :) -eric On Sun, Jul 17, 2016 at 10:57 AM, Guido van Rossum wrote: > This PEP is now accepted for inclusion in Python 3.6. Martin, > congratulations! Someone (not me) needs to review and commit your > changes, before September 12, when the 3.6

Re: [Python-Dev] Convert from unsigned long long to PyLong

2016-07-22 Thread Eric Snow
On Fri, Jul 22, 2016 at 3:02 AM, Stefan Ring wrote: > So to sum this up, you claim that PyLong_FromUnsignedLongLong can > somehow produce a number larger than the value range of a 64 bit > number (0x10180). I have a hard time believing this. Perhaps I misunderstood your meaning, but P

Re: [Python-Dev] The devguide is now hosted on GitHub

2016-07-22 Thread Eric Snow
Thanks for doing all this, Brett. :) -eric On Fri, Jul 22, 2016 at 2:04 PM, Brett Cannon wrote: > https://github.com/python/devguide > > I have also moved all issues over as well and hooked up Read The Docs so > that there's a mirror which is always up-to-date (vs. > docs.python.org/devguide whi

[Python-Dev] A Pseudo-Post-Mortem (not dead yet) on my Multi-Core Python Project.

2016-09-06 Thread Eric Snow
I'm not anticipating much discussion on this, but wanted to present a summary of my notes from the project I proposed last year and have since tabled. http://ericsnowcurrently.blogspot.com/2016/09/solving-mutli-core-python.html -eric ___ Python-Dev mail

[Python-Dev] PEP 468 ready for pronouncement.

2016-09-08 Thread Eric Snow
see: https://github.com/python/peps/blob/master/pep-0468.txt With the introduction of the compact dict implementation for CPython 3.6, PEP 468 becomes no more than a change to the language reference. I've adjusted the PEP to specify use of an ordered mapping rather than exactly OrderedDict. Thank

Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Eric Snow
On Sep 10, 2016 11:00, "Nick Coghlan" wrote: > > On 11 September 2016 at 03:08, Guido van Rossum wrote: > > So I'm happy to continue thinking about this, but I expect this is not > > such a big deal as you fear. Anyway, let's see if someone comes up > > with a more convincing argument by beta 2!

Re: [Python-Dev] PEP520 and absence of __definition_order__

2016-09-10 Thread Eric Snow
On Sep 10, 2016 10:11, "Guido van Rossum" wrote: > > Thanks for bringing this up. I think it's definitely possible to argue > either way. I think what happened before was that I approved > __definition_order__ because I wasn't expecting dict to be ordered by > default. Now that Inada Naoki's patch

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-12 Thread Eric Snow
On Mon, Sep 12, 2016 at 4:46 PM, Ethan Furman wrote: > Does anyone have a short explanation of the interaction between hash > randomization and this new always ordered dict? Why doesn't one make the > other useless? Before 3.6, dict iteration was based on the hash table, which varies based on th

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-14 Thread Eric Snow
On Wed, Sep 14, 2016 at 7:33 AM, INADA Naoki wrote: > I'll improve OrderedDict after dict in 3.6 is stable enough. +1 and if it's done carefully we could even utilize the pure Python OrderedDict and get rid of odictobject.c (and fold dict-common.h back into dictobject.c). We'd need to leave the

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-15 Thread Eric Snow
On Sep 15, 2016 06:06, "Serhiy Storchaka" wrote: > Python 3.5: 10 loops, best of 3: 33.5 msec per loop > Python 3.6: 10 loops, best of 3: 37.5 msec per loop > > These results look surprisingly and inexplicably to me. I expected that even if there is some performance regression in the lookup or mod

Re: [Python-Dev] Python 3.6 dict becomes compact and gets a private version; and keywords become ordered

2016-09-20 Thread Eric Snow
On Tue, Sep 20, 2016 at 5:11 AM, INADA Naoki wrote: > But both of OrderedDict and lru_cache improvements can't be in 3.6 > since 3.6 is beta now. > I'll try it after 3.6rc1. When you do, make sure you keep in mind the performance constraints of *all* the OrderedDict methods. The constraints are

Re: [Python-Dev] Update to PEP 1 re: content type

2016-10-13 Thread Eric Snow
On Thu, Oct 13, 2016 at 3:59 PM, Barry Warsaw wrote: > I don't understand the question: isn't the whole point of reST that it's a > easily readable markup language? Just post the reST! +1 -eric ___ Python-Dev mailing list Python-Dev@python.org https:/

Re: [Python-Dev] Benchmarking Python and micro-optimizations

2016-10-20 Thread Eric Snow
On Thu, Oct 20, 2016 at 4:56 AM, Victor Stinner wrote: > Hi, > > Last months, I worked a lot on benchmarks. I ran benchmarks, analyzed > results in depth (up to the hardware and kernel drivers!), I wrote new > tools and enhanced existing tools. This is a massive contribution. Thanks! > All thes

Re: [Python-Dev] why _PyGen_Finalize(gen) propagates close() to _PyGen_yf() ?

2017-03-28 Thread Eric Snow
On Mon, Mar 20, 2017 at 11:30 AM, Oleg Nesterov wrote: > Hello, > > Let me first clarify, I do not claim this is a bug, I am trying to learn > python and now I trying to understand yield-from. Given that you haven't gotten a response here, you may want to take this over to the core-mentors...@pyt

Re: [Python-Dev] 2017 Python Language Summit coverage

2017-05-25 Thread Eric Snow
On Thu, May 25, 2017 at 7:03 AM, Jake Edge wrote: > > Hola python-dev, > > Thanks to Larry and Barry, I was able to sit in on the Python Language > Summit again this year. The start of the coverage for that is now > available. > > [snipped] > > Hopefully I captured things reasonably well -- if yo

Re: [Python-Dev] startup time repeated? why not daemon

2017-07-20 Thread Eric Snow
On Thu, Jul 20, 2017 at 11:53 AM, Jim J. Jewett wrote: > I agree that startup time is a problem, but I wonder if some of the pain > could be mitigated by using a persistent process. > > [snip] > > Is it too hard to create a daemon server? > Is the communication and context switch slower than a new

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-24 Thread Eric Snow
On Thu, Aug 24, 2017 at 7:52 AM, Barry Warsaw wrote: > Guido van Rossum wrote: >> On Tue, Aug 22, 2017 at 7:12 PM, Nathaniel Smith wrote: >> >> I worry that that's going to lead more people astray thinking this has >> something to do with contextlib, which it really doesn't (it's much more >> clo

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-24 Thread Eric Snow
On Thu, Aug 24, 2017 at 8:23 AM, Yury Selivanov wrote: > Contrary to scoping, the programmer is much less likely to deal with > Execution Context. How often do we use "threading.local()"? This is an important point. PEP 550 is targeting library authors, right? Most folks will not be interactin

Re: [Python-Dev] PEP 550 v3 naming

2017-08-24 Thread Eric Snow
On Thu, Aug 24, 2017 at 5:37 AM, Nick Coghlan wrote: > Migrating a (variant of a) naming subthread from python-ideas over to > here, does the following sound plausible to anyone else?: > > ContextLocal - read/write access API (via get()/set() methods) > ContextLocalNamespace - active mappi

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-25 Thread Eric Snow
On Fri, Aug 25, 2017 at 8:18 AM, Yury Selivanov wrote: > Another idea: > > 1. We alter PyModule to make it possible to add properties (descriptor > protocol, or we implement custom __getattr__). I think we can make it > so that only sys module would be able to actually use it, so it's not > going

Re: [Python-Dev] Scope, not context? (was Re: PEP 550 v3 naming)

2017-08-25 Thread Eric Snow
On Fri, Aug 25, 2017 at 9:10 AM, Barry Warsaw wrote: > It’s ideas like this that do make me think of scopes when talking about > global state and execution contexts. I understand that the current PEP 550 > invokes an explicit separate namespace, Right. The observation that PEP 550 proposes a

Re: [Python-Dev] PEP 550 v4

2017-08-26 Thread Eric Snow
Thanks for the update. Comments in-line below. -eric On Fri, Aug 25, 2017 at 4:32 PM, Yury Selivanov wrote: > [snip] > > This PEP adds a new generic mechanism of ensuring consistent access > to non-local state in the context of out-of-order execution, such > as in Python generators and coroutin

Re: [Python-Dev] PEP 550 v4

2017-08-26 Thread Eric Snow
On Sat, Aug 26, 2017 at 11:19 AM, Yury Selivanov wrote: > This is similar to namedtuples, and nobody really complains about them. FWIW, there are plenty of complaints on python-ideas about this (and never a satisfactory solution). :) That said, I don't think it is as big a deal here since the t

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Eric Snow
On Sat, Aug 26, 2017 at 3:09 PM, Nathaniel Smith wrote: > You might be interested in these notes I wrote to motivate why we need > a chain of namespaces, and why simple "async task locals" aren't > sufficient: > > https://github.com/njsmith/pep-550-notes/blob/master/dynamic-scope.ipynb Thanks

Re: [Python-Dev] PEP 550 v4

2017-08-28 Thread Eric Snow
On Sat, Aug 26, 2017 at 10:31 AM, Yury Selivanov wrote: > On Sat, Aug 26, 2017 at 9:33 AM, Sven R. Kunze wrote: > [..] >> Why not the same interface as thread-local storage? This has been the >> question which bothered me from the beginning of PEP550. I don't understand >> what inventing a new wa

Re: [Python-Dev] PEP 550 V5

2017-09-01 Thread Eric Snow
Nice working staying on top of this! Keeping up with discussion is arguably much harder than actually writing the PEP. :) I have some comments in-line below. -eric On Fri, Sep 1, 2017 at 5:02 PM, Yury Selivanov wrote: > [snip] > > Abstract > > > [snip] > > Rationale > = > > [

Re: [Python-Dev] [RFC] Removing pure Python implementation of OrderedDict

2017-09-05 Thread Eric Snow
On Tue, Sep 5, 2017 at 1:38 AM, INADA Naoki wrote: > Like that, how about removing OrderedDict Pure Python implementation > from stdlib and require it to implementation? -1 Like Antoine, I consider the pure Python implementation to be valuable. Furthermore, the pure Python implementation is the

[Python-Dev] Re: Clarification regarding Stable ABI and _Py_*

2021-12-08 Thread Eric Snow
On Wed, Dec 8, 2021 at 2:23 AM Petr Viktorin wrote: > That really depends on what function we'd want to remove. There are > usually alternatives to deleting things, but the options depend on the > function. If we run out of other options we can make the function always > fail or make it leak memor

[Python-Dev] Re: Clarification regarding Stable ABI and _Py_*

2021-12-09 Thread Eric Snow
On Thu, Dec 9, 2021 at 1:56 AM Petr Viktorin wrote: > It's possible to remove them just like _PyObject_GC_Malloc was removed, > but check that it was unusable (e.g. not called from public macros) in > all versions of Python from 3.2 up to now. That's what I expected. Thanks. > Could you check i

[Python-Dev] Explicit markers for special C-API situations (re: Clarification regarding Stable ABI and _Py_*)

2021-12-09 Thread Eric Snow
(replying to https://mail.python.org/archives/list/python-dev@python.org/message/OJ65FPCJ2NVUFNZDXVNK5DU3R3JGLL3J/) On Wed, Dec 8, 2021 at 10:06 AM Eric Snow wrote: > What about the various symbols listed in Misc/stable_abi.txt that were > accidentally added to the limited API? Can w

[Python-Dev] Re: Explicit markers for special C-API situations (re: Clarification regarding Stable ABI and _Py_*)

2021-12-09 Thread Eric Snow
On Thu, Dec 9, 2021, 11:26 Petr Viktorin wrote: > I'll not get back to CPython until Tuesday, but I'll add a quick note > for now. It's a bit blunt for lack of time; please don't be offended. > Not at all. :) The tooling is a secondary concern to my point. Mostly, I wish the declarations in th

[Python-Dev] my plans for subinterpreters (and a per-interpreter GIL)

2021-12-14 Thread Eric Snow
Hi all, I'm still hoping to land a per-interpreter GIL for 3.11. There is still a decent amount of work to be done but little of it will require solving any big problems: * pull remaining static globals into _PyRuntimeState and PyInterpreterState * minor updates to PEP 554 * finish up the last c

[Python-Dev] subinterpreters and their possible impact on large extension projects

2021-12-14 Thread Eric Snow
One of the open questions relative to subinterpreters is: how to reduce the amount of work required for extension modules to support them? Thanks to Petr Viktorin for a lot of work he's done in this area (e.g. PEP 489)! Extensions also have the option to opt out of subinterpreter support. Howeve

[Python-Dev] "immortal" objects and how they would help per-interpreter GIL

2021-12-14 Thread Eric Snow
Most of the work toward interpreter isolation and a per-interpreter GIL involves moving static global variables to _PyRuntimeState or PyInterpreterState (or module state). Through the effort of quite a few people, we've made good progress. However, many globals still remain, with the majority bei

[Python-Dev] Re: subinterpreters and their possible impact on large extension projects

2021-12-14 Thread Eric Snow
Yeah, no (mutable) global state at the C level. It would also be good to implement multi-phase init (PEP 489), but I don't expect that to require much work itself. -eric On Tue, Dec 14, 2021 at 4:04 PM Brett Cannon wrote: > > > > On Tue, Dec 14, 2021 at 9:41 AM Eric Snow wr

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-14 Thread Eric Snow
On Tue, Dec 14, 2021 at 4:09 PM Brett Cannon wrote: > There's also the concern of memory usage if these immortal objects are never > collected. > > But which objects are immortal? You only listed None, True, and False. > Otherwise assume/remember I'm management and provide a list and/or link of

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Wed, Dec 15, 2021 at 2:50 AM Pablo Galindo Salgado wrote: > One thing to consider: ideally, inmortal objects should not participate in > the GC. There is nothing inheritly wrong if they do but we would need to > update the GC (and therefore add more branching in possible hot paths) to > deal

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Wed, Dec 15, 2021 at 2:42 AM Christian Heimes wrote: > Would it be possible to write the Py_INCREF() and Py_DECREF() macros in > a way that does not depend on branching? For example we could use the > highest bit of the ref count as an immutable indicator and do something like As Antoine point

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Tue, Dec 14, 2021 at 11:19 AM Eric Snow wrote: > The idea of objects that never get deallocated isn't new and has been > explored here several times. Not that long ago I tried it out by > setting the refcount really high. That worked. Around the same time > Eddie Elizond

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Wed, Dec 15, 2021 at 4:03 AM Victor Stinner wrote: > The last time I saw a benchmark on immortal object, it was clearly 10% > slower overall on the pyperformance benchmark suite. That's a major > slowdown. Yes, I plan on benchmarking the change as soon as we can run pyperformance on main. > >

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Wed, Dec 15, 2021 at 8:16 AM Skip Montanaro wrote: > It might be worth (re)reviewing Sam Gross's nogil effort to see how he > approached this: Yeah, there is good info in there. -eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscr

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-15 Thread Eric Snow
On Wed, Dec 15, 2021 at 6:16 AM Antoine Pitrou wrote: > Did you try to take into account the envisioned project for adding a > "complete" GC and removing the GIL? Yeah. I was going to start a separate thread about per-interpreter GIL vs. no-gil, but figured I was already pushing my luck with 3 s

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Tue, Dec 14, 2021 at 11:19 AM Eric Snow wrote: > There is one solution that would help both of the above in a nice way: > "immortal" objects. FYI, here are some observations that came up during some discussions with the "faster-cpython" team today: * immortal objec

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Eric Snow
On Wed, Dec 15, 2021 at 12:18 PM Chris Angelico wrote: > Sorry if this is a dumb question, but would it be possible to solve > that last point with an immortal arena [1] from which immortal objects > could be allocated? None/True/False could be allocated there, but so > could anything that is more

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-15 Thread Eric Snow
On Tue, Dec 14, 2021 at 10:12 AM Eric Snow wrote: > * it is fully backward compatible and the C-API is essentially unaffected Hmm, this is a little misleading. It will definitely be backward incompatible for extension modules that don't work under multiple subinterpreters (or rely on th

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-16 Thread Eric Snow
On Thu, Dec 16, 2021 at 2:48 AM Petr Viktorin wrote: > But does the sign bit need to stay intact, and do we actually need to > rely on the immortal bit to always be set for immortal objects? > If the refcount rolls over to zero, an immortal object's dealloc could > bump it back and give itself ano

[Python-Dev] Re: subinterpreters and their possible impact on large extension projects

2021-12-16 Thread Eric Snow
On Thu, Dec 16, 2021 at 4:34 AM Antoine Pitrou wrote: > As a data point, in PyArrow, we have a bunch of C++ code that interacts > with Python but doesn't belong in a particular Python module. That C++ > code can of course have global state, including perhaps Python objects. Thanks for that examp

[Python-Dev] Re: Static types and subinterpreters running in parallel

2021-12-16 Thread Eric Snow
On Thu, Dec 16, 2021 at 10:54 AM Guido van Rossum wrote: > > Eric has been looking into this. It's probably the only solution if we can't > get immutable objects. Yep. I've investigated the following approach (for the objects exposed in the public and limited C-API): * add a pointer field to P

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2022-01-05 Thread Eric Snow
On Wed, Jan 5, 2022, 15:02 Trent Nelson wrote: > I thought that was pretty interesting. Potentially many, many upper > bits for the taking. The code also had some logic that would int 3 > as soon as a 32-bit refcnt overflowed, and that never hit either > (obviously, based on the numbers above).

[Python-Dev] Re: Python no longer leaks memory at exit

2022-01-27 Thread Eric Snow
On Thu, Jan 27, 2022 at 8:40 AM Victor Stinner wrote: > tl; dr Python no longer leaks memory at exit on the "python -c pass" command > ;-) Thanks to all for the effort on this! Would it be worth adding a test to make sure we don't start leaking memory again? -eric _

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-02 Thread Eric Snow
On Wed, Feb 2, 2022 at 3:41 PM Eric Snow wrote: > I'd also like to actually get rid of _Py_IDENTIFIER(), along with > other related API including ~14 (private) C-API functions. FTR, here is the (private/internal) C-API affected by getting rid of _Py_IDENTIFIER(): * 21 C-API fun

[Python-Dev] Moving away from _Py_IDENTIFIER().

2022-02-02 Thread Eric Snow
I'm planning on moving us to a simpler, more efficient alternative to _Py_IDENTIFIER(), but want to see if there are any objections first before moving ahead. Also see https://bugs.python.org/issue46541. _Py_IDENTIFIER() was added in 2011 to replace several internal string object caches and to su

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Eric Snow
On Wed, Feb 2, 2022 at 11:50 PM Inada Naoki wrote: > It would be nice to provide something similar to _PY_IDENTIFIER, but > designed (and documented) for 3rd party modules like this. I suppose I'd like to know what the value of _Py_IDENTIFIER() is for 3rd party modules. They can already use PyUn

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Eric Snow
On Thu, Feb 3, 2022 at 6:46 AM Ronald Oussoren wrote: > Although my gut feeling is that adding a the CI check you mention is good > enough and adding the tooling for generating code isn’t worth the additional > complexity. Yeah, I came to the same conclusion. :) -eric ___

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Eric Snow
On Thu, Feb 3, 2022 at 7:17 AM Victor Stinner wrote: > In the top 5000 PyPI projects, I found 11 projects using them: > [snip[ > They use the these 17 functions: Thanks! That is super helpful. > If the _Py_IDENTIFIER() API is removed, it would be *nice* to provide > a migrate path (tool?) to he

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Eric Snow
On Thu, Feb 3, 2022 at 7:26 AM Victor Stinner wrote: > In bpo-39465, I made the _PyUnicode_FromId() compatible with running > sub-interpreters in parallel (one GIL per interpreter). > > A "static" PyUnicodeObject would have to share the reference count > between sub-interpreters, whereas Py_INCREF

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Eric Snow
On Thu, Feb 3, 2022 at 4:01 PM Guido van Rossum wrote: > Why not read through some of that code and see what they are doing with it? Yep, I'm planning on it. > I imagine one advantage is that _Py_IDENTIFIER() can be used entirely local > to a function. Yeah, they'd have to put something like t

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-04 Thread Eric Snow
On Fri, Feb 4, 2022 at 8:21 AM Stefan Behnel wrote: > Correct. We (intentionally) have our own way to intern strings and do not > depend on CPython's identifier framework. You're talking about __Pyx_StringTabEntry (and __Pyx_InitString())? -eric ___ Py

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-04 Thread Eric Snow
On Thu, Feb 3, 2022 at 3:49 PM Eric Snow wrote: > I suppose I'd like to know what the value of _Py_IDENTIFIER() is for > 3rd party modules. Between Guido, Victor, Stefan, and Sebastian, I'm getting the sense that a public replacement for _Py_IDENTIFER() would be worth pursuing.

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-04 Thread Eric Snow
On Fri, Feb 4, 2022, 16:03 Guido van Rossum wrote: > I wonder if a better solution than that PR wouldn't be to somehow change > the implementation of _Py_IDENTIFIER() to do that, > Yeah, I had the same realization today. I'm going to try it out. -eric > ___

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-07 Thread Eric Snow
On Fri, Feb 4, 2022 at 8:25 PM Eric Snow wrote: > On Fri, Feb 4, 2022, 16:03 Guido van Rossum wrote: >> I wonder if a better solution than that PR wouldn't be to somehow change the >> implementation of _Py_IDENTIFIER() to do that, > > Yeah, I had the same realization t

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2022-02-08 Thread Eric Snow
On Wed, Dec 15, 2021 at 10:15 AM Eric Snow wrote: > Yes, I plan on benchmarking the change as soon as we can run > pyperformance on main. I just ran the benchmarks and the PR makes CPython 4% slower. See https://github.com/python/cpython/pull/19474#issuecomment-1032944709.

[Python-Dev] PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-15 Thread Eric Snow
al Objects, Using a Fixed Refcount Author: Eric Snow , Eddie Elizondo Discussions-To: python-dev@python.org Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 10-Feb-2022 Python-Version: 3.11 Post-History: Resolution: Abstract Under this proposal, any object may be marked a

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-16 Thread Eric Snow
On Wed, Feb 16, 2022 at 12:37 AM Inada Naoki wrote: > +1 for overall idea. Great! > > Also note that "fork" isn't the only operating system mechanism > > that uses copy-on-write semantics. > > Could you elaborate? mmap, maybe? > [snip[ > So if you know how to get benefit from CoW without fork, I

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-16 Thread Eric Snow
Thanks for the feedback. My responses are inline below. -eric On Wed, Feb 16, 2022 at 6:36 AM Petr Viktorin wrote: > Thank you very much for writing this down! It's very helpful to see a > concrete proposal, and the current state of this idea. > I like the change, That's good to hear. :) > b

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-16 Thread Eric Snow
On Wed, Feb 16, 2022 at 12:14 PM Kevin Modzelewski wrote: > fwiw Pyston has immortal objects, though with a slightly different goal and > thus design [1]. I'm not necessarily advocating for our design (it makes most > sense if there is a JIT involved), but just writing to report our experience

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-16 Thread Eric Snow
On Wed, Feb 16, 2022 at 2:41 PM Terry Reedy wrote: > > * the naive implementation shows a 4% slowdown > > Without understanding all the benefits, this seems a bit too much for > me. 2% would be much better. Yeah, we consider 4% to be too much. 2% would be great. Performance-neutral would be eve

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-17 Thread Eric Snow
On Wed, Feb 16, 2022 at 10:43 PM Jim J. Jewett wrote: > I suggest being a little more explicit (even blatant) that the particular > details of: > [snip] > are not only Cpython-specific, but are also private implementation details > that are expected to change in subsequent versions. Excellent p

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-17 Thread Eric Snow
Again, thanks for the reply. It's helpful. My further responses are inline below. -eric On Thu, Feb 17, 2022 at 3:42 AM Petr Viktorin wrote: > > Agreed. However, what behavior do users expect and what guarantees do > > we make? Do we indicate how to interpret the refcount value they > > rece

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-18 Thread Eric Snow
On Wed, Feb 16, 2022 at 8:45 PM Inada Naoki wrote: > Is there any common tool that utilize CoW by mmap? > If you know, please its link to the PEP. > If there is no common tool, most Python users can get benefit from this. Sorry, I'm not aware of any, but I also haven't researched the topic much.

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount"

2022-02-18 Thread Eric Snow
On Wed, Feb 16, 2022 at 11:06 AM Larry Hastings wrote: > I experimented with this at the EuroPython sprints in Berlin years ago. I > was sitting next to MvL, who had an interesting observation about it. Classic MvL! :) > He suggested(*) all the constants unmarshalled as part of loading a modu

[Python-Dev] PEP 683: "Immortal Objects, Using a Fixed Refcount" (round 2)

2022-02-18 Thread Eric Snow
Thanks to all those that provided feedback. I've worked to substantially update the PEP in response. The text is included below. Further feedback is appreciated. -eric PEP: 683 Title: Immortal Objects, Using a Fixed Refcount Author: Eric Snow , Eddie Eli

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount" (round 2)

2022-02-22 Thread Eric Snow
from it, assuming we can reduce the performance penalty to acceptable levels. For a recent example, see https://mail.python.org/archives/list/python-dev@python.org/message/B77BQQFDSTPY4KA4HMHYXJEV3MOU7W3X/. > On Sat, Feb 19, 2022 at 4:52 PM Eric Snow wrote: > > > > Reducing CPU Cache In

[Python-Dev] Re: PEP 683: "Immortal Objects, Using a Fixed Refcount" (round 2)

2022-02-22 Thread Eric Snow
Thanks for the responses. I've replied inline below. -eric On Mon, Feb 21, 2022 at 9:11 AM Petr Viktorin wrote: > > On 19. 02. 22 8:46, Eric Snow wrote: > > Thanks to all those that provided feedback. I've worked to > > substantially update the PEP in response.

<    1   2   3   4   5   6   >