[Python-Dev] Re: Comments on PEP 558 (Defined semantics for locals() )
Unfortunately, the simplifications you propose would be backwards incompatible - it's existing behaviour that there's a real shared dict (even on optimised frames) where arbitrary extra attributes can be stored (even though they don't become accessible as Python variables). I don't want to make frame objects any bigger than they already are, so the natural solution is to store the mapping proxy as `f_locals`, and then bypass the proxy in order to make `PyEval_GetLocals` still "work" (at least as well as it ever did). PyObject_GetAttr(string) also doesn't do that same thing as the proposed C functions, since it invokes the Python descriptor machinery. (Note that the discussion at https://discuss.python.org/t/pep-558-defined-semantics-for-locals/2936/ is more up to date than the PEP text where the C API is concerned) The reference to tracing mode dependent semantics puzzles me, as that was removed in December: https://github.com/python/peps/commit/54888058ce8ad5257114652d9b41e8d1237b8ef9#diff-5abd04ea7e619670b52d61883873e784 Cheers, Nick. On Wed, 22 Jan 2020 at 21:59, Mark Shannon wrote: > > Hi, > > First of all I want to say that I'm very much in favour of the general > idea behind PEP 558. Defined semantics is always better than undefined > semantics :) > > However, I think there are a few changes needed: > > 1. Don't add anything to the C API, please. > Frame attributes can be accessed via `PyObject_GetAttr[String]`. > > 2. Don't make the behaviour dependent on whether "tracing" is turned on. > Doing so forces debuggers to use sys.settrace which is horribly > slow. It also makes the implementation more complex, and has no benefit > AFAICT. > > 3. Don't store write-through proxies in the frame, but make proxies > retain a reference to the frame. This would reduce the size and > complexity of code for handling frames. Clean up of the frame would > occur naturally via reference count when all proxies have been reclaimed. > > > > The proposed implementation is hard to reason about and I am not > confident that it will not introduce some new subtle bugs to replace the > ones it seeks to remove. > Any implementation that has functions with "Borrow" and "BreakCycles" in > their names makes me nervous. > > > A simpler conceptual model, which I believe could be made reliable, > would be: > > No change for non-function frames (as PEP 558 currently proposes). > > Each access to `frame.f_locals` (for function frames) should return a > new proxy object. > > This proxy object would have dict-like, write-through semantics for > variables in the frame. For keys that do not match variable names, an > exception would be raised. This means that all proxies for a single > frame will have value equivalence; object equivalence is not needed. > I.e. for a frame `f`, `f.f_locals == f.f_locals` would be True, even > though `f.f_locals is f.f_locals` would be False. > > Cheers, > Mark. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/TSHCB4ZHC4XUVGJ3MX4D7HCHMTB6WROD/ > Code of Conduct: http://python.org/psf/codeofconduct/ -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/X5KMI4CPJSXZCI4342FNHX4T6R2PBMGN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: extern "C" { ... } in Include/cpython/*.h
On Tue, 28 Jan 2020 at 20:43, Petr Viktorin wrote: > On 2020-01-27 21:42, Skip Montanaro wrote: > > I just noticed that Nick migrated the guts of Include/frameobject.h to > > include/cpython/frameobject.h. It's not clear to me that the latter > > should be #include'd directly from anywhere other than > > Include/frameobject.h. If that's the case, does the extern "C" stuff > > still need to be replicated in the lower level file? Won't the scope > > of the extern "C" block in Include/frameobject.h be active at the > > lower level? > > > > Whatever the correct answer is, I suspect the same constraints should > > apply to all Include/cpython/*.h files. > > You're correct: > - The #ifndef/#error make a clear point that a include/cpython/*.h file > is only to be included as part of its include/*.h > - The inner extern "C" block overrides the outer one. If they're the > same, it's useless. I hadn't thought about this before (and presumably Victor hadn't either when he did the initial set of header splits), but yeah, I think you're right - the inner C-style linking declarations are redundant, given the macro definitions checks that ensure these files are always going to be included inside the outer C-style linking declarations. > But I'd call it a minor issue – a style issue, even. Probably not worth > worrying about. A quick code search says there are only 18 header files affected by this at the moment - if we were to clean those up, then any future file splits would presumably copy that improved style, rather than perpetuating the redundancy. So call it a +0 from me for tidying them up. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QP23KZZKISBIIP76AHPX7WFKMOOX2UCQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] pickle.reduce and deconstruct functions
This was [posted on -ideas][1], but apparently many people didn't see it
because of the GMane migration going on at exactly the same time. At any rate,
Antoine Pitrou suggested it should be discussed on -dev instead. And this gives
me a chance to edit it (apparently it was markdown-is enough to confuse
Hyperkitty and turn into a mess).
Pickling uses an extensible protocol that lets any class determine how its
instances can be deconstructed and reconstructed. Both `pickle` and `copy` use
this protocol, but it could be useful more generally. Unfortunately, to use it
more generally requires relying on undocumented details. I think we should
expose a couple of helpers to fix that:
# Return the same (shallow) reduction tuple that pickle.py, copy.py, and
_pickle.c would use
pickle.reduce(obj) -> (callable, args[, state[, litems[, ditem[,
statefunc)
# Return a callable and arguments to construct a (shallow) equivalent object
# Raise a TypeError when that isn't possible
pickle.deconstruct(obj) -> callable, args, kw
So, why do you want these?
There are many cases where you want to "deconstruct" an object if possible. For
example:
* Pattern matching depends on being able to deconstruct objects like this
* Auto-generating a `__repr__` as suggested in [Chris Angelico's -ideas
thread][2].
* Quick&dirty REPL stuff, and deeper reflection stuff using
`inspect.signature` and friends.
Of course not every type tells `pickle` what to do in an appropriate way that
we can use, but a pretty broad range of types do, including (I think; I haven't
double-checked all of them) `@dataclass`, `namedtuple`, `@attr.s`, many builtin
and extension types, almost all reasonable types that use `copyreg`, and any
class that pickles via the simplest customization hook `__getnewargs[_ex]__`.
That's more than enough to be useful.
And, just as important, it won't (except in intentionally pathological cases)
give us a false positive, where a type is correctly pickleable and we think we
can deconstruct it but the deconstruction is wrong. (For some uses, you are
going to want to fall back to heuristics that are often right but sometimes
misleadingly wrong, but I don't think the `pickle` module should offer anything
like that. Maybe `inspect` should, but I'm not proposing that here.)
The way to get the necessary information isn't fully documented, and neither is
the way to interpret it. And I don't think it _should_ be documented, because
it changes every so often, and for good reasons; we don't want anyone writing
third-party code that relies on those details. Plus, a different Python
implementation might conceivably do it differently. Public helpers exposed from
`pickle` itself won't have those problems.
Here's a first take at the code.
def reduce(obj, proto=pickle.DEFAULT_PROTOCOL):
"""reduce(obj) -> (callable, args[, state[, litems[, ditem[,
statefunc)
Return the same reduction tuple that the pickle and copy modules use
"""
cls = type(obj)
if reductor := copyreg.dispatch_table.get(cls):
return reductor(obj) # Note that this is not a special method call
(not looked up on the type)
if reductor := getattr(obj, "__reduce_ex__"):
return reductor(proto)
if reductor := getattr(obj, "__reduce__"):
return reductor() raise TypeError(f"{cls.__name__} objects are not
reducible")
def deconstruct(obj):
"""deconstruct(obj) -> callable, args, kwargs
callable(_args, **kwargs) will construct an equivalent object
"""
reduction = reduce(obj)
# If any of the optional members are included, pickle/copy has to
# modify the object after construction, so there is no useful single
# call we can deconstruct to.
if any(reduction[2:]):
raise TypeError(f"{type(obj).__name__} objects are not
deconstrutable")
func, args,_ _ = reduction
# Many types (including @dataclass, namedtuple, and many builtins)
# use copyreg.__newobj__ as the constructor func. The args tuple is
# the type (or, when appropriate, some other registered
# constructor) followed by the actual args. However, any function
# with the same name will be treated the same way (because under the
# covers, this is optimized to a special opcode).
if func.__name__ == "__newobj__":
return args[0], args[1:], {}
# (Mainly only) used by types that implement __getnewargs_ex__ use
# copyreg.__newobj_ex__ as the constructor func. The args tuple
# holds the type, *args tuple, and **kwargs dict. Again, this is
# special-cased by name.
if func.__name__ == "__newobj_ex__":
return args
# If any other special copyreg functions are added in the future,
# this code won't k
