[Python-Dev] Re: Comments on PEP 558

[Mark Shannon]

On 28/07/2021 1:03 am, Nick Coghlan wrote:



On Wed, 28 Jul 2021, 1:50 am Mark Shannon, > wrote:


Hi Nick,

On 27/07/2021 2:29 pm, Nick Coghlan wrote:
 >
 >
 > The reference documentation should be precise as well, since that is
 > what other implementations will be following.
 >
 > What semantics do you feel are left unspecified?
 >
 >> The documentation needs to explain the behavior to the majority of
 >> users, but the PEP should be worded so that it can be implemented
 >> correctly from just reading the PEP.
 >>
 >> There is no reason to remove the sugested documentation changes, but
 >> they should be secondary to the more formal specification.
 >
 > I don't know what you want to see on this front.
 >
 > The current PEP seems complete to me, except in relation to the
 > specifics of the proxy behaviour, which it deliberately leaves
 > underspecified (mostly because my current implementation sucks in a
 > few areas, and I really don't want to enshrine those limitations in
 > the language spec).

All the above were mostly just suggestions, the format of the PEP is up
to you.

But please, do not underspecify anything. I need to maintain this,
as do
the PyPy and MicroPython developers. We need this to be precise.


You still haven't said what you consider underspecified.


You said it was underspecified, not me. Quoting you from a few lines up
"which it deliberately leaves underspecified"



Perhaps read the implementation and tell me which parts of what the 
tests are covering you want to see replicated in the PEP text?



 >
 >> Resolving the issues with tracing mode behaviour
 >> 
 >>
 >> According to this section the `f_locals` object
(_PyFastLocalsProxy_Type
 >> in C) will have *two* fields.
 >> It should only have *one*, the pointer to the underlying frame
object.
 >
 > That's a good point - fast_refs doesn't reference the frame
object, so
 > it's safe to store it on the frame object and reference it from the
 > individual proxies.
 >
 > Switching to that approach also has the advantage that every proxy
 > after the first is much cheaper, since they don't need to build the
 > fast refs map again.
 >
 >> In order to maximize backwards compatibility and, more importantly,
 >> avoid the synchronization issues that lead to obscure bugs, all the
 >> state of the `f_locals` object should be kept on the frame.
 >>
 >> Any changes to `f_locals` should be instantly visible to any
thread both
 >> through other `f_locals` objects (for the same frame) and the
underlying
 >> local variable (if it exists).
 >
 > [snip proxy method implementation sketch] >
 > This gets the gist of how the implementation works, but it isn't
quite
 > that simple for a few reasons:
 >
 > 1. The proxy needs to offer the same algorithmic complexity as dict
 > APIs, so doing O(n) linear searches of C arrays and Python tuples
 > inside O(1) lookup operations isn't OK (hence the fastrefs mapping to
 > allow for O(1) resolution of variable names to cell references and
 > local variable array offsets)

Linear searches are faster for small arrays, and there is nothing
stopping you adding a helper map of names->indexes for large functions.
Provided the mapping from name to local index is O(1), then the whole
operation is O(1).


That helper map already exists in the fast refs mapping. Skipping it for 
small functions would be a future optimisation opportunity that doesn't 
seem necessary in the initial implementation.


fastref is a mapping from names to values. I am suggesting a mapping 
from names to indexes in the locals array. That is not the same thing.
The mapping from names to indexes is fixed at compile time and cannot 
get out of sync.





You really don't need much extra machinery to maintain O(1) behavior
(which no one cares about, they just care about overall performance).


Yes, the only extra machinery needed for working with individual keys is 
the fast refs mapping.


It's other aspects of the mapping API that benefit from the continuing 
presence of the  frame cache (which is maintained on the frame, just as 
it is today, NOT separately in each proxy).


If you can give me a reliable O(1) implementation of 
"len(frame.f_locals)" that doesn't rely on up to date cache of:

* which local and cell variable names are currently bound to values
* which extra variables have been added via either proxies or 
PyEval_GetLocals()


1. You don't need a O(1) implementation of len(frame.f_locals); no one 
cares.


2. Your implementation isn't O(1) either, because you are ignoring the 
costs of updating your cache.






 > 2. The proxy needs to deal with *SIX* kinds of possi

[Python-Dev] Re: Comments on PEP 558

[Nick Coghlan]

On Wed, 28 Jul 2021, 8:13 pm Mark Shannon,  wrote:

>
>
> On 28/07/2021 1:03 am, Nick Coghlan wrote:
> >
> >
> > On Wed, 28 Jul 2021, 1:50 am Mark Shannon,  > > wrote:
> >
> > Hi Nick,
> >
> > On 27/07/2021 2:29 pm, Nick Coghlan wrote:
> >  >
> >  >
> >  > The reference documentation should be precise as well, since that
> is
> >  > what other implementations will be following.
> >  >
> >  > What semantics do you feel are left unspecified?
> >  >
> >  >> The documentation needs to explain the behavior to the majority
> of
> >  >> users, but the PEP should be worded so that it can be implemented
> >  >> correctly from just reading the PEP.
> >  >>
> >  >> There is no reason to remove the sugested documentation changes,
> but
> >  >> they should be secondary to the more formal specification.
> >  >
> >  > I don't know what you want to see on this front.
> >  >
> >  > The current PEP seems complete to me, except in relation to the
> >  > specifics of the proxy behaviour, which it deliberately leaves
> >  > underspecified (mostly because my current implementation sucks in
> a
> >  > few areas, and I really don't want to enshrine those limitations
> in
> >  > the language spec).
> >
> > All the above were mostly just suggestions, the format of the PEP is
> up
> > to you.
> >
> > But please, do not underspecify anything. I need to maintain this,
> > as do
> > the PyPy and MicroPython developers. We need this to be precise.
> >
> >
> > You still haven't said what you consider underspecified.
>
> You said it was underspecified, not me. Quoting you from a few lines up
> "which it deliberately leaves underspecified"
>

I said the proxy was under specified. The entire frame API is a CPython
implementation detail that some other implementations emulate, so I'm OK
with that.

You were claiming the *locals()* behaviour was under specified, so I want
to know what is unclear.


> Perhaps read the implementation and tell me which parts of what the
> > tests are covering you want to see replicated in the PEP text?
> >
> >
> >  >
> >  >> Resolving the issues with tracing mode behaviour
> >  >> 
> >  >>
> >  >> According to this section the `f_locals` object
> > (_PyFastLocalsProxy_Type
> >  >> in C) will have *two* fields.
> >  >> It should only have *one*, the pointer to the underlying frame
> > object.
> >  >
> >  > That's a good point - fast_refs doesn't reference the frame
> > object, so
> >  > it's safe to store it on the frame object and reference it from
> the
> >  > individual proxies.
> >  >
> >  > Switching to that approach also has the advantage that every proxy
> >  > after the first is much cheaper, since they don't need to build
> the
> >  > fast refs map again.
> >  >
> >  >> In order to maximize backwards compatibility and, more
> importantly,
> >  >> avoid the synchronization issues that lead to obscure bugs, all
> the
> >  >> state of the `f_locals` object should be kept on the frame.
> >  >>
> >  >> Any changes to `f_locals` should be instantly visible to any
> > thread both
> >  >> through other `f_locals` objects (for the same frame) and the
> > underlying
> >  >> local variable (if it exists).
> >  >
> >  > [snip proxy method implementation sketch] >
> >  > This gets the gist of how the implementation works, but it isn't
> > quite
> >  > that simple for a few reasons:
> >  >
> >  > 1. The proxy needs to offer the same algorithmic complexity as
> dict
> >  > APIs, so doing O(n) linear searches of C arrays and Python tuples
> >  > inside O(1) lookup operations isn't OK (hence the fastrefs
> mapping to
> >  > allow for O(1) resolution of variable names to cell references and
> >  > local variable array offsets)
> >
> > Linear searches are faster for small arrays, and there is nothing
> > stopping you adding a helper map of names->indexes for large
> functions.
> > Provided the mapping from name to local index is O(1), then the whole
> > operation is O(1).
> >
> >
> > That helper map already exists in the fast refs mapping. Skipping it for
> > small functions would be a future optimisation opportunity that doesn't
> > seem necessary in the initial implementation.
>
> fastref is a mapping from names to values. I am suggesting a mapping
> from names to indexes in the locals array. That is not the same thing.
> The mapping from names to indexes is fixed at compile time and cannot
> get out of sync.
>


Umm, please read the PEP again. You are completely confusing the fast refs
mapping with the frame cache. They're not the same thing at all.


> >
> >
> > You really don't need much extra machinery to maintain O(1) behavior
> > (which no one cares about, th

[Python-Dev] Re: 咨询python的相关问题(Consult python related issues)

[Terry Reedy]

On 7/28/2021 6:46 AM, C wrote:

Hello, I would like to consult related issues encountered when reading 
the python language reference manual,


Ask question on python-list, which is about *using* python.
https://mail.python.org/mailman/listinfo/python-list

python-dev is for discussion of *developing* future versions of Python 
and CPython.


Both are English only.

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WKEITIR5S65NSOVWGPV7FPUDXGVHXTBG/
Code of Conduct: http://python.org/psf/codeofconduct/