[Python-Dev] Re: Problems with dict subclassing performance

[Federico Salerno]

"Pretendere" in Italian means "to demand", it's a false friend with the 
English "pretend". I don't know whether Marco is Italian (the false 
friend might also be there between Spanish or whatever other romance 
language he speaks and English, for all I know). From a native Italian 
speaker's perspective, what he meant was very clear to me, but it's also 
clear that an English speaker with no experience of Italian would not be 
expected to understand the meaning necessarily.


Either way, from an outsider's perspective this whole bickering over 
such a small thing seems unfit for a list where adults talk about 
technical details. It seems to me that someone should swallow their 
pride and let this thread drop once and for all, it's not bringing 
anything useful or relevant.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I4MMDDKQRK5ZW4MJRHUMYNLTLJ5AQLMW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problems with dict subclassing performance

[Jeff Allen]

On 16/08/2021 08:41, Federico Salerno wrote:
"Pretendere" in Italian means "to demand", it's a false friend with 
the English "pretend". I don't know whether Marco is Italian (the 
false friend might also be there between Spanish or whatever other 
romance language he speaks and English, for all I know). From a native 
Italian speaker's perspective, what he meant was very clear to me, but 
it's also clear that an English speaker with no experience of Italian 
would not be expected to understand the meaning necessarily.


I reached the same conclusion with the help of: 
https://it.wiktionary.org/wiki/pretendere#Etimologia_/_Derivazione . Pop 
open the translations (Traduzioine). English has diverged from the 
latin, as explained elsewhere on the thread.


It's all easily explained by a misunderstanding of Steven's remark about 
upvotes (to agree with the answer provided by Monica) and subsequent 
frustration in being unable to make oneself understood.




Either way, from an outsider's perspective this whole bickering over 
such a small thing seems unfit for a list where adults talk about 
technical details. It seems to me that someone should swallow their 
pride and let this thread drop once and for all, it's not bringing 
anything useful or relevant.



Quite possibly. But I've offered a grown-up response in a separate post.

--

Jeff Allen

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZYRNFVFCF2AJQUMUHXHWYBDZMZGLTB4I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problems with dict subclassing performance

[Chris Angelico]

On Mon, Aug 16, 2021 at 5:44 PM Federico Salerno  wrote:
>
> "Pretendere" in Italian means "to demand", it's a false friend with the
> English "pretend". I don't know whether Marco is Italian (the false
> friend might also be there between Spanish or whatever other romance
> language he speaks and English, for all I know). From a native Italian
> speaker's perspective, what he meant was very clear to me, but it's also
> clear that an English speaker with no experience of Italian would not be
> expected to understand the meaning necessarily.
>

If THAT'S what it is, why couldn't someone say so earlier???

If Marco had simply reworded it saying "I demand your immediate
excuses", we would at least have understood.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H2MJPJVFY2MOAVK5MONFKYEDOAYP3MBO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problems with dict subclassing performance

[Jeff Allen]

On 06/08/2021 20:29, Marco Sulla wrote:
I've done an answer on SO about why subclassing `dict` makes the 
subclass so much slower than `dict`. The answer is interesting: 
https://stackoverflow.com/questions/59912147/why-does-subclassing-in-python-slow-things-down-so-much 


What do you think about? 
I have spent a lot of time reading typeobject.c over the years I've been 
looking at an alternative implementation. It's quite difficult to 
follow, and full of tweaks for special circumstances. So I'm impressed 
with the understanding that "user2357112 supports Monica" brings to the 
subject. (Yes, I want to call them Monica too, but I don't think that's 
their actual name. ) I don't think I understand it better than they but 
here's my reading of that, informed by my reading of typeobject.c, in 
case it helps.


When a built-in type like dict is defined in C, pointers to its C 
implementation functions are hard-coded into slots in the type object. 
In order to make each appear as a method to Python, a descriptor is 
created when building the type that delegates to the slot (so 
sq_contains generates a descriptor __contains__ in the dictionary of the 
type.


Conversely, if in a sub-class you define __contains__, then the type 
builder will insert a function pointer in the slot of the new type that 
arranges a call to __contains__. This will overwrite whatever was in the 
slot.


In a C implementation, you can also define methods (by creating a 
PyMethodDef the tp_methods table) that become descriptors in the 
dictionary of the type. You would not normally define both a C function 
to place in the slot *and* the corresponding method via a PyMethodDef. 
If you do, the version from the dictionary of the type will win the 
slot, *unless* you mark the method definition (in its PyMethodDef) as 
METH_COEXIST.


This exception is used in the special case of dict (and hardly anywhere 
else but set I think). I assume this is because some important code 
calls __contains__ via the descriptor, rather than via the slot (which 
would be quicker), and because an explicit definition is faster than a 
descriptor created automatically to wrap the slot.


Now, when you create a sub-class, the table of slots is copied first, 
then the type is checked for definitions of special methods, and these 
are allowed to overwrite the slot, unless they are slot wrappers on the 
same function pointer the slot already contains. I think at this point 
the slot is re-written to contain a wrapper on __contains__, which has 
been inherited from dict.__contains__, because it isn't a *slot wrapper* 
on the same function. For example:


>>> dict.__contains__
   
>>> str.__contains__
   

>>> class S(str): pass

>>> S.__contains__
   
>>> D.__contains__
   

I think that when filling the slots of a sub-class, one could check for 
the METH_COEXIST flag at the point one checks to see whether the 
definition from look-up on the type is a PyWrapperDescr on the same 
pointer. One might have to know that the slot and descriptor come from 
the same base. I'm not suggesting this would be worthwhile.


FYI, in the approach I am toying with, the slot wrapper descriptor is 
always created from the function definition, then the slot is filled 
from the available definitions by lookup. Defining __contains__ twice 
would be impossible or an error. I think this has the semantics required 
by Python, but we'll have to wait for proof.


-- Jeff Allen 
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZKMVZ5M3V76SOZH7FOURQ66VFZQY2BTG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making code object APIs unstable

[Steve Dower]

On 8/16/2021 12:47 AM, Guido van Rossum wrote:
My current proposal is to issue a DeprecationWarning in PyCode_New() and 
PyCode_NewWithPosArgs(), which can be turned into an error using a 
command-line flag. If it's made an error, we effectively have (B); by 
default, we have (A).


Then in 3.13 we can drop them completely.


We definitely had legitimate use cases come up when adding positional 
arguments (hence the new API, rather than breaking the existing one, 
which was the first attempt at adding the feature).


I don't recall exactly what they are (perhaps Pablo does, or they may be 
in email/issue archives), but since they exist, presumably they are 
useful and viable _despite_ the bytecode varying between releases. This 
suggests there's probably a better API we should add at the same time - 
possibly some kind of unmarshalling or cloning-with-updates function?


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NXSTTN3CVCGAYSCUH24OEZZFIBOEJRBD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 558, the simplest thing I could come up with

[Brett Cannon]

On Sat, Aug 14, 2021 at 8:01 PM Nick Coghlan  wrote:

> Bringing the public record up to date with a brief off-list discussion
> between Mark, Nathaniel and I:
>
> * Mark hasn't convinced me that getting rid of the frame value cache
> entirely for optimised frames is a good idea, so he's going to write that
> proposal up as a competing PEP. Once it has been drafted and is ready for
> review, he will request the SC assign a PEP delegate.
> * On the PEP 558 front, Mark's proposal has highlighted a few
> inefficiencies in my reference implementation, where the code still
> implicitly updates the frame value cache in cases where the cache being up
> to date may not matter to the proxy API client. So I'll be working on
> another iteration of the implementation that ensures each caching proxy
> instance (at worst) only pays the O(N) cache refresh price on the first
> less than O(N) operation that relies on the cache being up to date, rather
> than paying it every time "f_locals" is retrieved from the frame object.
>
> We still have plenty of time before 3.11b1, so we expect it will be a
> month or two before the two proposals are in a position to be compared
> directly.
>

I think I can speak for the SC by saying, "please, take as much time as you
want" 😉. We are still trying to get out for our PEP review backlog as it
is.

-Brett


>
> Cheers,
> Nick.
>
> On Fri, 30 Jul 2021, 5:25 pm Nick Coghlan,  wrote:
>
>>
>>
>> On Fri, 30 Jul 2021, 2:30 pm Nathaniel Smith,  wrote:
>>
>>>
>>> >
>>> > For [proxy] versus [snapshot], a lot depends on what we think of
>>> changing the semantics of exec(). [proxy] is definitely more consistent and
>>> elegant, and if we could go back in time I think it's
>>> what we'd have done from the start. Its compatibility is maybe a bit
>>> worse than [snapshot] on non-exec() cases, but this seems pretty minor
>>> overall (it often doesn't matter, and if it does just write
>>> dict(locals()) instead of locals(), like you would in non-function
>>> scope). But the change in exec() semantics is an actual language
>>> change, even though it may not affect much real code, so that's what
>>> stands out for me.
>>>
>>> I *think* (please correct me if I'm wrong) that what that calls
>>> [PEP-minus-tracing] is now corresponds to the current PEP draft, and
>>> [proxy] corresponds to Mark's draft at the beginning of this thread?
>>>
>>
>> At the locals() level, PEP 558 is now [snapshot] (due to your original
>> analysis showing that it was strictly better than what I had at the time),
>> while Mark's draft is indeed [proxy].
>>
>> Cheers,
>> Nick.
>>
>>
>>
>>> -n
>>>
>>> --
>>> Nathaniel J. Smith -- https://vorpus.org
>>>
>> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EU2TU6R2HDV6NLZQ56MSDIRMG6EHSOJO/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U4K7OFXLQTNW667XAVEJPTPFSDTSWXB4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] python-dev thread w/ Marco Sulla

[Brett Cannon]

https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/

I can't be objective with Marco as I believe we have recorded issues with
him previously (as with Steven if you take Marco's initial side with this).

The thing that pushed me over the edge to report this was
https://mail.python.org/archives/list/python-dev@python.org/message/O3JB3FE33KMT3OHZCVH3XO6VNJTGH5NL/
.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TX44HOPXBSHDPA5ZXCAUGWTQDHCTD723/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Problems with dict subclassing performance

[Brett Cannon]

On Sun, Aug 15, 2021 at 2:55 PM Marco Sulla 
wrote:

> On Sun, 15 Aug 2021 at 23:33, Tim Peters 
> wrote:ople have said now, including me, they had no idea what
> > you meant.by "I pretend your immediate excuses". It's not a complaint
> > that it's expressed inelegantly, but that they can't make _any_ sense
> > of it. By my count, this is at least the second time you've declined
> > to explain what you meant, but instead implied the person who said
> > they couldn't understand it was being dishonest.
>
> I repeat, even the worst AI will understand from the context what I
> meant. But let me do a very rude example:
>

Rude examples are never necessary and are not acceptable (don't forget we
have kids who participate on this mailing list).

I have referred the whole thread to the Conduct WG so they can settle who
was out of line. Otherwise I advise everyone to mute this thread.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZVWHZJQU3QTL3GVRLHQHP4WZJWZP2O2P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Antoine Pitrou]

On Mon, 16 Aug 2021 09:47:13 -0700
Brett Cannon  wrote:
> https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/
> 
> I can't be objective with Marco as I believe we have recorded issues with
> him previously (as with Steven if you take Marco's initial side with this).
> 
> The thing that pushed me over the edge to report this was
> https://mail.python.org/archives/list/python-dev@python.org/message/O3JB3FE33KMT3OHZCVH3XO6VNJTGH5NL/
> .
> 

I think a private appeal to the two or three participants to remain
cool, nice and polite would be welcome here.  Also an appeal to
think twice before posting... (is what you're going to say really going
to benefit the community?)

Regards

Antoine.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OUKC6WY5VCAD2ZIKQZQUHOP5LUTSDMF7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making code object APIs unstable

[Guido van Rossum]

On Mon, Aug 16, 2021 at 9:30 AM Steve Dower  wrote:

> On 8/16/2021 12:47 AM, Guido van Rossum wrote:
> > My current proposal is to issue a DeprecationWarning in PyCode_New() and
> > PyCode_NewWithPosArgs(), which can be turned into an error using a
> > command-line flag. If it's made an error, we effectively have (B); by
> > default, we have (A).
> >
> > Then in 3.13 we can drop them completely.
>
> We definitely had legitimate use cases come up when adding positional
> arguments (hence the new API, rather than breaking the existing one,
> which was the first attempt at adding the feature).
>
> I don't recall exactly what they are (perhaps Pablo does, or they may be
> in email/issue archives), but since they exist, presumably they are
> useful and viable _despite_ the bytecode varying between releases. This
> suggests there's probably a better API we should add at the same time -
> possibly some kind of unmarshalling or cloning-with-updates function?
>

I presume the use cases are essentially some variant of the .replace() API
that exists at the Python level. At the C level you would get all fields
from an existing code object and pass them to PyCode_New[WithPosArgs]
except for e.g. the co_filename field. Unfortunately those use cases will
still break if there are any try blocks in the code or if the
endline/column info is needed. Also you can't access any of the code
object's fields without the internal API (this wasn't always so). So I
think it's different now.


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K2PUYK6MHXILWCBDWZZIUORIALI6SP4A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Kyle Stanley]

I can agree with the general premise of what Antoine is saying, but to me
even as a non-participant, the following quote from the thread Brett linked
seems a clear CoC violation:

I repeat, even the worst AI will understand from the context what I meant.
> But let me do a very rude example:
>
> What if I said to Steven "I pretend immediate suck my $%#$"? Do you think
> you and the others will not understand the sense? :D
>

This at least indicates a lengthy timeout is needed, imo.

On Mon, Aug 16, 2021 at 1:01 PM Antoine Pitrou  wrote:

> On Mon, 16 Aug 2021 09:47:13 -0700
> Brett Cannon  wrote:
> >
> https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/
> >
> > I can't be objective with Marco as I believe we have recorded issues with
> > him previously (as with Steven if you take Marco's initial side with
> this).
> >
> > The thing that pushed me over the edge to report this was
> >
> https://mail.python.org/archives/list/python-dev@python.org/message/O3JB3FE33KMT3OHZCVH3XO6VNJTGH5NL/
> > .
> >
>
> I think a private appeal to the two or three participants to remain
> cool, nice and polite would be welcome here.  Also an appeal to
> think twice before posting... (is what you're going to say really going
> to benefit the community?)
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/OUKC6WY5VCAD2ZIKQZQUHOP5LUTSDMF7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L5SEEI5DLPU2UMGERSKTQCMVCG4TAAQI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Nathaniel Smith]

Was this post intended to go to python-dev or...?

On Mon, Aug 16, 2021 at 9:53 AM Brett Cannon  wrote:
>
> https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/
>
> I can't be objective with Marco as I believe we have recorded issues with him 
> previously (as with Steven if you take Marco's initial side with this).
>
> The thing that pushed me over the edge to report this was 
> https://mail.python.org/archives/list/python-dev@python.org/message/O3JB3FE33KMT3OHZCVH3XO6VNJTGH5NL/.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/TX44HOPXBSHDPA5ZXCAUGWTQDHCTD723/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K3HEVE3HD5Y3DNB5ARF7V4USF6FWXYMU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Nathan C. Fox]

Yes, it was intended to go to python-dev, even though it's not about Python
development. It's part of a discussion about a pretty hostile and off-topic
thread that has unfolded over the last week on this mailing list.

https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/

Nathan Fox

On Mon, Aug 16, 2021 at 3:54 PM Nathaniel Smith  wrote:

> Was this post intended to go to python-dev or...?
>
> On Mon, Aug 16, 2021 at 9:53 AM Brett Cannon  wrote:
> >
> >
> https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/
> >
> > I can't be objective with Marco as I believe we have recorded issues
> with him previously (as with Steven if you take Marco's initial side with
> this).
> >
> > The thing that pushed me over the edge to report this was
> https://mail.python.org/archives/list/python-dev@python.org/message/O3JB3FE33KMT3OHZCVH3XO6VNJTGH5NL/
> .
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/TX44HOPXBSHDPA5ZXCAUGWTQDHCTD723/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/K3HEVE3HD5Y3DNB5ARF7V4USF6FWXYMU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AVRSOKRSHQAAJUIB4LDPHAOMMTGTXEXG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Jonathan Goble]

On Mon, Aug 16, 2021 at 4:04 PM Nathan C. Fox 
wrote:

> Yes, it was intended to go to python-dev, even though it's not about
> Python development. It's part of a discussion about a pretty hostile and
> off-topic thread that has unfolded over the last week on this mailing list.
>

Brett's original post here reads as though it was intended as a private
email to a moderator, though.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J6KOYHBSK4V2CV2LKRAP62467IERKY42/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: python-dev thread w/ Marco Sulla

[Brett Cannon]

On Mon, Aug 16, 2021 at 1:37 PM Jonathan Goble  wrote:

> On Mon, Aug 16, 2021 at 4:04 PM Nathan C. Fox 
> wrote:
>
>> Yes, it was intended to go to python-dev, even though it's not about
>> Python development. It's part of a discussion about a pretty hostile and
>> off-topic thread that has unfolded over the last week on this mailing list.
>>
>
> Brett's original post here reads as though it was intended as a private
> email to a moderator, though.
>

Not that it really makes a difference at this point, but no, it was not
meant for this mailing list.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AGJLFJMBZLOL2HF2FE5HGZBDNHNGMO44/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making code object APIs unstable

[Nick Coghlan]

On Tue, 17 Aug 2021, 4:30 am Guido van Rossum,  wrote:

> On Mon, Aug 16, 2021 at 9:30 AM Steve Dower 
> wrote:
>
>> On 8/16/2021 12:47 AM, Guido van Rossum wrote:
>> > My current proposal is to issue a DeprecationWarning in PyCode_New()
>> and
>> > PyCode_NewWithPosArgs(), which can be turned into an error using a
>> > command-line flag. If it's made an error, we effectively have (B); by
>> > default, we have (A).
>> >
>> > Then in 3.13 we can drop them completely.
>>
>> We definitely had legitimate use cases come up when adding positional
>> arguments (hence the new API, rather than breaking the existing one,
>> which was the first attempt at adding the feature).
>>
>> I don't recall exactly what they are (perhaps Pablo does, or they may be
>> in email/issue archives), but since they exist, presumably they are
>> useful and viable _despite_ the bytecode varying between releases. This
>> suggests there's probably a better API we should add at the same time -
>> possibly some kind of unmarshalling or cloning-with-updates function?
>>
>
> I presume the use cases are essentially some variant of the .replace() API
> that exists at the Python level. At the C level you would get all fields
> from an existing code object and pass them to PyCode_New[WithPosArgs]
> except for e.g. the co_filename field. Unfortunately those use cases will
> still break if there are any try blocks in the code or if the
> endline/column info is needed. Also you can't access any of the code
> object's fields without the internal API (this wasn't always so). So I
> think it's different now.
>

A cloning-with-replacement API that accepted the base code object and the
"safe to modify" fields could be a good complement to the API deprecation
proposal.

Moving actual "from scratch" code object creation behind the Py_BUILD_CORE
guard with an underscore prefix on the name would also make sense, since it
defines a key piece of the compiler/interpreter boundary.

Cheers,
Nick.

P.S. Noting an idea that won't work, in case anyone else reading the thread
was thinking the same thing: a "PyType_FromSpec" style API won't help here,
as the issue is that the compiler is now doing more work up front and
recording that extra info in the code object for the interpreter to use.
There is no way to synthesise that info if it isn't passed to the
constructor, as it isn't intrinsically recorded in the opcode sequence.


>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/K2PUYK6MHXILWCBDWZZIUORIALI6SP4A/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YIKXHLNR4FYP7ZU5PUTRIBNHENIXEM7A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making code object APIs unstable

[Guido van Rossum]

On Mon, Aug 16, 2021 at 4:44 PM Nick Coghlan  wrote:

> [...]
> A cloning-with-replacement API that accepted the base code object and the
> "safe to modify" fields could be a good complement to the API deprecation
> proposal.
>

Yes (I forgot to mention that).


> Moving actual "from scratch" code object creation behind the Py_BUILD_CORE
> guard with an underscore prefix on the name would also make sense, since it
> defines a key piece of the compiler/interpreter boundary.
>

Yeah, we have _PyCode_New() for that.


> Cheers,
> Nick.
>
> P.S. Noting an idea that won't work, in case anyone else reading the
> thread was thinking the same thing: a "PyType_FromSpec" style API won't
> help here, as the issue is that the compiler is now doing more work up
> front and recording that extra info in the code object for the interpreter
> to use. There is no way to synthesise that info if it isn't passed to the
> constructor, as it isn't intrinsically recorded in the opcode sequence.
>

That's the API style that _PyCode_New() uses (thanks to Eric who IIRC
pushed for this and implemented it). You gave me an idea now: the C
equivalent to .replace() could use the same input structure; one can leave
fields NULL that should be copied from the original unmodified.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NWYMCDAMS4YRJ7ESXNWQ6MIBSRAZEXEM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Making code object APIs unstable

[Gregory P. Smith]

Doing a search of a huge codebase (work), the predominant user of
PyCode_New* APIs appears to be checked in Cython generated code (in all
sorts of third_party OSS projects). It's in the boilerplate that Cython
extensions make use of via it's __Pyx_PyCode_New macro.
https://github.com/cython/cython/blob/master/Cython/Utility/ModuleSetupCode.c#L470

I saw very few non-Cython uses.  There are some, but at a very quick first
glance they appear simple - easy enough to reach out to the projects with a
PR to update their code.

The Cython use will require people to upgrade Cython and regenerate their
code before they can use the Python version that changes these. That is not
an uncommon thing for Cython. It's unfortunate that many projects on ship
generated sources rather than use Cython at build time, but that isn't
_our_ problem to solve. The more often we change internal APIs that things
depend on, the more people will move their projects towards doing the right
thing with regards to either not using said APIs or rerunning an up to date
code generator as part of their build instead of checking in generated
unstable API using sources.

-gps


On Mon, Aug 16, 2021 at 8:04 PM Guido van Rossum  wrote:

> On Mon, Aug 16, 2021 at 4:44 PM Nick Coghlan  wrote:
>
>> [...]
>> A cloning-with-replacement API that accepted the base code object and the
>> "safe to modify" fields could be a good complement to the API deprecation
>> proposal.
>>
>
> Yes (I forgot to mention that).
>
>
>> Moving actual "from scratch" code object creation behind the
>> Py_BUILD_CORE guard with an underscore prefix on the name would also make
>> sense, since it defines a key piece of the compiler/interpreter boundary.
>>
>
> Yeah, we have _PyCode_New() for that.
>
>
>> Cheers,
>> Nick.
>>
>> P.S. Noting an idea that won't work, in case anyone else reading the
>> thread was thinking the same thing: a "PyType_FromSpec" style API won't
>> help here, as the issue is that the compiler is now doing more work up
>> front and recording that extra info in the code object for the interpreter
>> to use. There is no way to synthesise that info if it isn't passed to the
>> constructor, as it isn't intrinsically recorded in the opcode sequence.
>>
>
> That's the API style that _PyCode_New() uses (thanks to Eric who IIRC
> pushed for this and implemented it). You gave me an idea now: the C
> equivalent to .replace() could use the same input structure; one can leave
> fields NULL that should be copied from the original unmodified.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NWYMCDAMS4YRJ7ESXNWQ6MIBSRAZEXEM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/67DMIW7NQE6M6LEPLANXKZQEFOFVPBBL/
Code of Conduct: http://python.org/psf/codeofconduct/