[Python-Dev] Re: Comments on PEP 558 (Defined semantics for locals() )

[Nick Coghlan]

On Mon., 10 Feb. 2020, 8:31 pm Mark Shannon,  wrote:

>
> On 08/02/2020 11:49 am, Nick Coghlan wrote:
> > Unfortunately, the simplifications you propose would be backwards
> > incompatible - it's existing behaviour that there's a real shared dict
> > (even on optimised frames) where arbitrary extra attributes can be
> > stored (even though they don't become accessible as Python variables).
> > I don't want to make frame objects any bigger than they already are,
> > so the natural solution is to store the mapping proxy as `f_locals`,
> > and then bypass the proxy in order to make `PyEval_GetLocals` still
> > "work" (at least as well as it ever did).
>
> The proposed changes in PEP 558 are also backwards incompatible.
> I thought that was the point. The current implementation is broken in
> weird ways and we want to fix that. Since we need to break backward
> compatibility anyway, why not do it in a way the makes the behaviour as
> well defined and maintainable as possible.
>

The changes at the Python level are *technically* incompatible, but
Nathaniel's review made a compelling case that the real world compatibility
problems were likely to be minimal, and in some cases would actually be
fixing latent defects in existing code.

I think that PEP 558, as it stands, is still a bit fragile because of
> the handling of cycles between the locals proxy and the frame.
>

Unfortunately, I'm not entirely sure there's any way to get rid of that
without getting rid of PyEval_GetLocals() completely, and that *wouldn't*
be a subtle break in the slightest (it's even part of the stable ABI).

Since that API returns a borrowed reference, the real reference has to live
somewhere, and the most natural place is the frame f_locals attribute (as
that's where it lives today).

And even if we *did* manage to resolve that dilemna, we then run into the
problem that we also need the frame object to hold the proxy because the C
level equivalent of accessing the attribute is just "frame->f_locals": it's
not an opaque struct, so that pointer is part of the public API.

I agree I should explain this aspect clearly in the PEP though (and likely
in some block comments in the implementation), as you're quite right that
the associated reference borrowing and cycle breaking code is thoroughly
nasty now that the namespace object isn't going to be a simple dictionary.

(Thinking out loud, though: something that might work is for each locals
proxy to use a common snapshot namespace, and store *that* on the frame,
exactly as we do today. That would replace the cycle in the current
implementation with multiple references to the common snapshot)


> >
> > PyObject_GetAttr(string) also doesn't do that same thing as the
> > proposed C functions, since it invokes the Python descriptor
> > machinery. (Note that the discussion at
> > https://discuss.python.org/t/pep-558-defined-semantics-for-locals/2936/
> > is more up to date than the PEP text where the C API is concerned)
>
> `PyObject_GetAttr("attr")` has the same semantics as the Python operator
> `x.attr` which is under the control of `type(x)`, in this case the frame
> object class. The descriptor protocol is irrelevant.
>

The now proposed C APIs don't include one to get access to the
write-through proxy, so you do indeed have to use PyObject_GetAttr for that.

The proposed APIs instead aim to make it possible to access the Python
locals without needing to acquire a frame object reference at all.


> >
> > The reference to tracing mode dependent semantics puzzles me, as that
> > was removed in December:
> >
> https://github.com/python/peps/commit/54888058ce8ad5257114652d9b41e8d1237b8ef9#diff-5abd04ea7e619670b52d61883873e784
> >
>
> That was my misreading. The behaviour of `f_locals` in the PEP is not
> very clear, as it is buried in the discussion of CPython changes.
> Could you add it to the proposal section?
>

Aye, I'll do that when clarifying the complications arising from wanting to
keep PyEval_GetLocals() and direct "frame->f_locals" access working pretty
closely to the way they have in the past, while still resolving the other
issues.

(And writing that sentence is what made me realise that you could be right
and it may be possible to make this work without needing to give the frame
object a reference to any of the write-through proxy objects)

Cheers,
Nick.


>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/53AHUQRPGHG7KUD4W4YPPHZKMXIXAAHU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 584: Add Union Operators To dict

[Brandt Bucher]

After a few days of thinking and experimenting, I’ve been convinced that `copy` 
(and also `__copy__`) is not the right protocol for what we want to do here. I 
believe that 584 can likely continue without subclass-preserving behavior, but 
that better behavior could perhaps could be added to *all* built-in types 
later, since it’s outside the scope of this PEP.

> My opinion is that Python built-in types make subclassing unnecessarily(?) 
> awkward to use, and I would like to see that change.

Yes! But, on further reflection, I don’t think this is the correct way of 
approaching it.

> For example, consider subclassing float. If you want your subclass to be 
> actually usable, you have to write a whole bunch of boilerplate, otherwise 
> the first time you perform arithmetic on your subclass, it will be converted 
> to a regular old float… This is painful and adds a great amount of friction 
> to subclassing.

`float` is a *perfect* example of the problems with the way things are 
currently, so let’s focus on this.

Currently, subclassing `float` requires ~30 overridden methods of repetitive 
(but non-trivial) boilerplate to get everything working right. However, calling 
the `float` equivalent of `dict.copy()` on the LHS before proceeding with the 
default implementation wouldn’t help us, because floats (like many built-in 
types) are immutable. So a new, plain, built-in `float` would still be returned 
by the default machinery. It doesn’t know how to construct a new, different 
instance of our subclass, and it can’t change one it’s already built.

This leads me to believe that we’re approaching the problem wrong. Rather than 
making a copy and working on it, I think the problem would be better served by 
a protocol that runs the default implementation, *then* calls some under hook 
on the subclass to build a new instance.

Let’s call this method `__build__`. I’m not sure what its arguments would look 
like, but it would probably need at least `self`, and an instance of the 
built-in base class (in this case a `float`), and return a new instance of the 
subclass based on the two. It would likely also need to work with `cls` instead 
of `self` for `classmethod` constructors like `dict.fromkeys`, or have a second 
hook for that case.

By subclassing `float` and defining `__build__` to something like this:

```
class MyFloat(float):
…
def __build__(self, result):
Return MyFloat(result, some_state=self.some_state)
…
```

I could now trust the built-in `float` machinery to try calling 
`lhs.__build__(result)` on the result that *would* have been returned *before* 
returning it. This is a simple example, but a protocol like this would work for 
mutables as well.

> A more pertinent example, from dict itself:

If `dict` *were* to grow more operators, they would likely be `^`, `&`, and 
`-`. You can consider the case of subclassing `set` or `frozenset`, since they 
currently has those. Calling `lhs.copy()` first before updating is fine for 
additive operations like `|`, but for subtractive operations like the others, 
this can be very bad for performance, especially if we’re now *required* to 
call them. Again, doing things the default way, and *then* constructing the 
appropriate subclass in an agreed-upon way seems like the path to take here.

> Changing all builtins is a big, backwards-incompatible change.

If implemented right, a system like the one described above (`__build__`) 
wouldn’t be backward-incompatible, as long as nobody was already using the name.

Just food for thought. I think this is a much bigger issue than PEP 584, but 
I'm convinced that the consistent status quo should prevail until a suitable 
solution for all types can be worked out (if ever).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7HNJ6RVVEX2VD37HYR5F5P5W2YAOMPDH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 584: Add Union Operators To dict

[Andrew Barnert via Python-Dev]

Brandt Bucher wrote:

> This leads me to believe that we’re approaching the problem wrong. Rather 
> than making a
> copy and working on it, I think the problem would be better served by a 
> protocol that runs
> the default implementation, then calls some under hook on the subclass to 
> build a
> new instance.
>
> Let’s call this method `__build__`. I’m not sure what its arguments would
> look like, but it would probably need at least `self`, and an instance of the
> built-in base class (in this case a `float`), and return a new instance of the
> subclass based on the two. It would likely also need to work with `cls` 
> instead
> of `self` for `classmethod` constructors like
> `dict.fromkeys`, or have a second hook for that case.

You can call `self.fromkeys`, and it works just like calling 
`type(self).fromkeys`.

The only real advantage of having a second hook is that it would simplify the 
most trivial cases—which are very common. In particular, probably 90% of 
subclasses of builtins are like Steven's `MyFloat` example—all you really want 
to do is call your constructor in place of the super's constructor, and if you 
have to call it with the result of your super's constructor instead, that's 
fine because `MyFloat(x)` on a `float` or `MyFloat` is equivalent to `x` 
anyway. So you could just write `__build_cls__ = __new__` and you're done. With 
only an instance-method version, you'd have to write `def __build__(self, 
other): return type(self)(other)`. Which isn't _terrible_ or anything, but as 
boilerplate that has to be added (probably without being understood) to 
hundreds of classes, it's not exactly ideal.

If there were a way to actually get your constructor called on the `__new__` 
arguments directly, without constructing the superclass instance first, that 
would be even better. Besides being more efficient (and that "more efficient" 
could actually be a big deal, because we're talking about every call to every 
operator dunder and many other methods on builtin needing to check this in 
addition to whatever else it does…), it would allow a trivial implementation on 
types that share their super's constructor signature but can't guarantee that 
`MyType(x) == x`. Even for cases like `defaultdict`, if you could supply a 
constructor, you'd be fine: `partial(self, self.default_factory)` can be used 
with the arguments to a `dict` construction call just as easily as it can be 
used with a `dict` itself.

But I'm not sure there is such a way. (Maybe the pickle/copy protocol can help 
here? Not sure without thinking it through more…)

> If implemented right, a system like the one described above
> (__build__) wouldn’t be backward-incompatible, as long as nobody 
> was already using the name.

Assuming the builtins don't grow `__build__` methods that use `cls` or 
`type(self)` (which is what you'd ideally want, but then you get the same 
massive backward-incompatibility problem we were trying to avoid…), it seems 
like we're adding possibly significant cost to everything (maybe not 
significant for `dict.__union__`, but maybe so for `int.__add__`) for a benefit 
that almost no code actually uses. Maybe the longterm benefit of everyone being 
able to drop those `MyFloat(…)` calls all over once they can require 3.10+ is 
worth the immediate and permanent cost to performance and implementation 
complexity, but I'm not sure.

(If there were an opt-in way to replace the super's construction call instead 
of post-hooking it, the cost might be reduced enough to change that 
calculation. But again, I'm not sure if there is such a way.)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AN5BE7GKCEOTEXD5I4YFKVSEDGBZRPN4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 584: Add Union Operators To dict

[Guido van Rossum]

So can we just finish PEP 584 without the .copy() call? Surely the more
general solution will not be ready for Python 3.9, while PEP 584 is nearly
done. (I'm also skeptical about a general solution, but I'd rather stay out
of that discussion for a while longer, and maybe you all come up with
something good.)

On Sun, Feb 16, 2020 at 8:46 AM Brandt Bucher 
wrote:

> After a few days of thinking and experimenting, I’ve been convinced that
> `copy` (and also `__copy__`) is not the right protocol for what we want to
> do here. I believe that 584 can likely continue without subclass-preserving
> behavior, but that better behavior could perhaps could be added to *all*
> built-in types later, since it’s outside the scope of this PEP.
>
> > My opinion is that Python built-in types make subclassing
> unnecessarily(?) awkward to use, and I would like to see that change.
>
> Yes! But, on further reflection, I don’t think this is the correct way of
> approaching it.
>
> > For example, consider subclassing float. If you want your subclass to be
> actually usable, you have to write a whole bunch of boilerplate, otherwise
> the first time you perform arithmetic on your subclass, it will be
> converted to a regular old float… This is painful and adds a great amount
> of friction to subclassing.
>
> `float` is a *perfect* example of the problems with the way things are
> currently, so let’s focus on this.
>
> Currently, subclassing `float` requires ~30 overridden methods of
> repetitive (but non-trivial) boilerplate to get everything working right.
> However, calling the `float` equivalent of `dict.copy()` on the LHS before
> proceeding with the default implementation wouldn’t help us, because floats
> (like many built-in types) are immutable. So a new, plain, built-in `float`
> would still be returned by the default machinery. It doesn’t know how to
> construct a new, different instance of our subclass, and it can’t change
> one it’s already built.
>
> This leads me to believe that we’re approaching the problem wrong. Rather
> than making a copy and working on it, I think the problem would be better
> served by a protocol that runs the default implementation, *then* calls
> some under hook on the subclass to build a new instance.
>
> Let’s call this method `__build__`. I’m not sure what its arguments would
> look like, but it would probably need at least `self`, and an instance of
> the built-in base class (in this case a `float`), and return a new instance
> of the subclass based on the two. It would likely also need to work with
> `cls` instead of `self` for `classmethod` constructors like
> `dict.fromkeys`, or have a second hook for that case.
>
> By subclassing `float` and defining `__build__` to something like this:
>
> ```
> class MyFloat(float):
> …
> def __build__(self, result):
> Return MyFloat(result, some_state=self.some_state)
> …
> ```
>
> I could now trust the built-in `float` machinery to try calling
> `lhs.__build__(result)` on the result that *would* have been returned
> *before* returning it. This is a simple example, but a protocol like this
> would work for mutables as well.
>
> > A more pertinent example, from dict itself:
>
> If `dict` *were* to grow more operators, they would likely be `^`, `&`,
> and `-`. You can consider the case of subclassing `set` or `frozenset`,
> since they currently has those. Calling `lhs.copy()` first before updating
> is fine for additive operations like `|`, but for subtractive operations
> like the others, this can be very bad for performance, especially if we’re
> now *required* to call them. Again, doing things the default way, and
> *then* constructing the appropriate subclass in an agreed-upon way seems
> like the path to take here.
>
> > Changing all builtins is a big, backwards-incompatible change.
>
> If implemented right, a system like the one described above (`__build__`)
> wouldn’t be backward-incompatible, as long as nobody was already using the
> name.
>
> Just food for thought. I think this is a much bigger issue than PEP 584,
> but I'm convinced that the consistent status quo should prevail until a
> suitable solution for all types can be worked out (if ever).
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/7HNJ6RVVEX2VD37HYR5F5P5W2YAOMPDH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lis

[Python-Dev] Re: PEP 584: Add Union Operators To dict

[Steven D'Aprano]

On Sun, Feb 16, 2020 at 02:12:48PM -0800, Guido van Rossum wrote:

> So can we just finish PEP 584 without the .copy() call?

Fine by me.

Thanks to everyone!

-- 
Steven
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H2TNHSCTSF3PUCN3KS7RY3A42UDMSMBK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: pickle.reduce and deconstruct functions

[Nick Coghlan]

I don't have anything to add on the side of the technical specifics, but I
do want to point out another benefit of adding these APIs: a significant
improvement in testability, since you will be able to test the reduction
and reconstruction behaviours directly, rather than having to infer what is
going on from the round trip behavior.

Cheers,
Nick.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XOJCW66YE3E5MNYL2Y6GMTBDJR7RMFJL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 584: Add Union Operators To dict

[Brandt Bucher]

> So can we just finish PEP 584 without the .copy() call?

Sounds good. I’ll update the implementation and open a PR to the PEPs repo 
later tonight. Thanks everybody!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IPSPMBI6YKWWLVJJT7V2O5X7HHMPD7FJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Regarding SSL installation in python

[Naga Chaitanya L]

Hi,
I want to know how to install ssl certificate in python. please give me
referral link.
thnk u
Regards!
chaitanya
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LS6OFKHCJILJIYRHRDCTHEMUHSAU47CE/
Code of Conduct: http://python.org/psf/codeofconduct/