[Python-Dev] Changing PySequence and PyMapping checks

[Bar Harel]

There's a long time issue of trying to differentiate mappings and sequences
in the C-API in a fast and reliable way.

Due to recent changes, we might be able to do so at last, by checking
tp_flags + str/bytes/bytearray which are considered unique.

This might be a breaking change in the Stable ABI promise but one that can
be considered a bug fix as the current behavior is lacking.

What do you think?

bpo: https://bugs.python.org/issue46376

Best regards,
Bar Harel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WTIKR5DGMZRKRRAZGFTD4EPG5ITFHOVS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Dict __missing__ consistency issues

[Bar Harel]

Hey guys,

I was just about to fix dict's get() to call __missing__ if a key doesn't
exist (before returning the default value) but realized that although
small, that patch can cause future issues.
Right now there's an inconsistency:

>>> from collections import UserDict
>>> class A(dict):
...  def __missing__(self, key):
...   print(key)
...
>>> class B(UserDict):
...  def __missing__(self, key):
...   print("UserDict", key)
...
>>> a = A()
>>> b = B()
>>> a.get(123)
>>> b.get(123)
UserDict 123
>>> a.get(123, "abc")
'abc'
>>> b.get(123, "abc")
UserDict 123

The reason for this inconsistency is because the Mapping abc and dict
behave differently.
Dict's get doesn't call __getitem__ which causes the call not to route to
__missing__.
MutableMapping's get calls __getitem__, which UserDict implements as a
check to __missing__ as well.

According to the doc, the specification requires dict's __getitem__ to call
__missing__. It doesn't say anything about get().

Should get() call __missing__?

If it does, things like defaultdict.get() might break. It will however be
more consistent with dict's specification.
If it doesn't, we expect Mapping to not care about __missing__ as it's only
a dict thing, which will require UserDict to override get(). Dict's get()
will need to receive a doc update as well stating __missing__ is not called.

Second question is: Is __missing__ only a dict thing, or is it part of the
Mapping ABC?

I would expect it to be a part of the Mapping ABC, with subclasses not
having to implement it. Right now it's not.

Looking forward for your inputs,
Bar Harel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JMNLIZJ6XBDCDR6PZWUN33PYOWO4ZFLP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Dict __missing__ consistency issues.

[Bar Harel]

Hey guys,

I was just about to fix dict's get() to call __missing__ if a key doesn't exist 
(before returning the default value) but realized that although small, that 
patch can cause future issues.
Right now there's an inconsistency:

>>> from collections import UserDict
>>> class A(dict):
...  def __missing__(self, key):
...   print(key)
... 
>>> class B(UserDict):
...  def __missing__(self, key):
...   print("UserDict", key)
... 
>>> a = A()
>>> b = B()
>>> a.get(123)
>>> b.get(123)
UserDict 123
>>> a.get(123, "abc")
'abc'
>>> b.get(123, "abc")
UserDict 123

The reason for this inconsistency is because the Mapping abc and dict behave 
differently.
Dict's get doesn't call __getitem__ which causes the call not to route to 
__missing__.
MutableMapping's get calls __getitem__, which UserDict implements as a check to 
__missing__ as well.

According to the doc, the specification requires dict's __getitem__ to call 
__missing__. It doesn't say anything about get().

Should get() call __missing__?

If it does, things like defaultdict.get() might break. It will however be more 
consistent with dict's specification.
If it doesn't, we expect Mapping to not care about __missing__ as it's only a 
dict thing, which will require UserDict to override get(). Dict's get() will 
need to receive a doc update as well stating __missing__ is not called.

Second question is: Is __missing__ only a dict thing, or is it part of the 
Mapping ABC?

I would expect it to be a part of the Mapping ABC, with subclasses not having 
to implement it. Right now it's not.

Looking forward for your inputs,
Bar Harel
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SDXOEMAEM6KQ3CQCJVBVRT5QNSPAVU6X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict __missing__ consistency issues.

[Bar Harel]

Sorry for the double post then :-)

As for get(). Current implementation of UserDict returns the default value
for get() if __missing__() raises KeyError. Which sounds reasonable to me.
After all, you would logically expect __missing__ to be called for get() as
you tried to get a non-existing value.

If __missing__ is only a dict thing, then I'll add a note to dict's .get()
method saying it doesn't call __missing__ for missing values.
I can also modify UserDict to behave in a consistent manner.

Does that make sense?

On Wed, Jan 8, 2020 at 7:42 PM Guido van Rossum  wrote:

> (Note: We received your email twice.)
>
> I don't think __missing__ should be called by get() -- get() already has a
> way to deal with missing keys, and making it use two different mechanisms
> would be weird (e.g. if get() calls __missing__, is the default value ever
> used?).
>
> To answer your second question, __missing__ is only a dict thing, it is
> not part of Mapping.
>
> On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:
>
>> Hey guys,
>>
>> I was just about to fix dict's get() to call __missing__ if a key doesn't
>> exist (before returning the default value) but realized that although
>> small, that patch can cause future issues.
>> Right now there's an inconsistency:
>>
>> >>> from collections import UserDict
>> >>> class A(dict):
>> ...  def __missing__(self, key):
>> ...   print(key)
>> ...
>> >>> class B(UserDict):
>> ...  def __missing__(self, key):
>> ...   print("UserDict", key)
>> ...
>> >>> a = A()
>> >>> b = B()
>> >>> a.get(123)
>> >>> b.get(123)
>> UserDict 123
>> >>> a.get(123, "abc")
>> 'abc'
>> >>> b.get(123, "abc")
>> UserDict 123
>>
>> The reason for this inconsistency is because the Mapping abc and dict
>> behave differently.
>> Dict's get doesn't call __getitem__ which causes the call not to route to
>> __missing__.
>> MutableMapping's get calls __getitem__, which UserDict implements as a
>> check to __missing__ as well.
>>
>> According to the doc, the specification requires dict's __getitem__ to
>> call __missing__. It doesn't say anything about get().
>>
>> Should get() call __missing__?
>>
>> If it does, things like defaultdict.get() might break. It will however be
>> more consistent with dict's specification.
>> If it doesn't, we expect Mapping to not care about __missing__ as it's
>> only a dict thing, which will require UserDict to override get(). Dict's
>> get() will need to receive a doc update as well stating __missing__ is not
>> called.
>>
>> Second question is: Is __missing__ only a dict thing, or is it part of
>> the Mapping ABC?
>>
>> I would expect it to be a part of the Mapping ABC, with subclasses not
>> having to implement it. Right now it's not.
>>
>> Looking forward for your inputs,
>> Bar Harel
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/SDXOEMAEM6KQ3CQCJVBVRT5QNSPAVU6X/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MVPF5AM2TDB65DRYA3UNXMN23IFICMOW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict __missing__ consistency issues.

[Bar Harel]

Well it depends on the consensus for __missing__, and the consensus for
get().

I believe modifying get() is out of the question as it will lead to
breakage. If so, whether it's a quirk or not doesn't matter as we can't
change it. Adding a comment to the docs should suffice, along with updating
the data model to explicitly state dict.__missing__ instead of
object.__missing__.

As for __missing__, if it's only a dict thing, I don't believe Mapping
should be aware of it's existence, which means the patch should center on
UserDict behaving differently (thus mimicking dict's quirky behavior).

Since we cannot override get() to access self.data, (as it will bypass and
break custom __getitem__) I think we might as well modify it to use the
__contains__ it already created.

A simple check of "if k in D __getitem__" will slow only UserDict, not all
mappings, and be an acceptable solution.

That's all as long as we accept the 2 facts that __missing__ is dict-only
and .get() will stay somewhat quirky and unexpected but documented.

On Wed, Jan 8, 2020 at 8:11 PM Guido van Rossum  wrote:

> Hum, it looks like the UserDict.get() behavior is an accident due to the
> way Mapping.get() is implemented. The comment there says
>
> 'D.get(k[,d]) -> D[k] if k in D, else d.  d defaults to None.'
>
> but the implementation just tries D[k] and catches KeyError. The
> UserDict.__contains__() implementation is explicitly code to avoid
> __missing__, since the default Mapping.__contains__() implementation *also*
> just calls D[k] and catches KeyError.
>
> More indication that the UserDict.get() behavior is an accident: in
> test_userdict.py there's a test for __missing__, test_missing() but it
> doesn't test get() at all, only __getitem__().
>
> How were you proposing to fix the situation? I could imagine changing
> Mapping.get() to call __contains__() first. That would slow down
> Mapping.get() slightly (the default implementation would end up calling
> __getitem__() twice if the key is present), but I don't think that's a
> grave concern (if people want it faster there are a zillion ways to do
> that).
>
> On Wed, Jan 8, 2020 at 9:49 AM Bar Harel  wrote:
>
>> Sorry for the double post then :-)
>>
>> As for get(). Current implementation of UserDict returns the default
>> value for get() if __missing__() raises KeyError. Which sounds
>> reasonable to me. After all, you would logically expect __missing__ to be
>> called for get() as you tried to get a non-existing value.
>>
>> If __missing__ is only a dict thing, then I'll add a note to dict's
>> .get() method saying it doesn't call __missing__ for missing values.
>> I can also modify UserDict to behave in a consistent manner.
>>
>> Does that make sense?
>>
>> On Wed, Jan 8, 2020 at 7:42 PM Guido van Rossum  wrote:
>>
>>> (Note: We received your email twice.)
>>>
>>> I don't think __missing__ should be called by get() -- get() already has
>>> a way to deal with missing keys, and making it use two different mechanisms
>>> would be weird (e.g. if get() calls __missing__, is the default value ever
>>> used?).
>>>
>>> To answer your second question, __missing__ is only a dict thing, it is
>>> not part of Mapping.
>>>
>>> On Wed, Jan 8, 2020 at 8:07 AM Bar Harel  wrote:
>>>
>>>> Hey guys,
>>>>
>>>> I was just about to fix dict's get() to call __missing__ if a key
>>>> doesn't exist (before returning the default value) but realized that
>>>> although small, that patch can cause future issues.
>>>> Right now there's an inconsistency:
>>>>
>>>> >>> from collections import UserDict
>>>> >>> class A(dict):
>>>> ...  def __missing__(self, key):
>>>> ...   print(key)
>>>> ...
>>>> >>> class B(UserDict):
>>>> ...  def __missing__(self, key):
>>>> ...   print("UserDict", key)
>>>> ...
>>>> >>> a = A()
>>>> >>> b = B()
>>>> >>> a.get(123)
>>>> >>> b.get(123)
>>>> UserDict 123
>>>> >>> a.get(123, "abc")
>>>> 'abc'
>>>> >>> b.get(123, "abc")
>>>> UserDict 123
>>>>
>>>> The reason for this inconsistency is because the Mapping abc and dict
>>>> behave differently.
>>>> Dict's get doesn't call __getitem__ which causes the call not to route
>>

[Python-Dev] Re: Dict __missing__ consistency issues.

[Bar Harel]

>
> [Bar Harel]
> > As for get(). Current implementation of UserDict returns the default
> value
> > for get() if __missing__() raises KeyError.  Which sounds reasonable to
> me.
> > After all, you would logically expect __missing__ to be called for get()
> > as you tried to get a non-existing value.
>
>
> [Ethan Furman]
> I disagree.  My understanding of the purpose behind get() is to get a value
> or return the default specified in the get() call, not to create a new key
> (which the default __missing__ does).
>

There is no default __missing__. That's what the __missing__ at defaultdict
does.

Like I said before: The fact __missing__ is not called for missing values
is a bit funny in my opinion, but it doesn't actually matter cause we can't
do anything about it without breaking lots of stuff (e.g. making
defaultdict.get worthless).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3HPG4NPXSVOB3Q5HE7TQE37JG6TUFLMC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Dict __missing__ consistency issues.

[Bar Harel]

Finished creating 2 issues on the meanwhile:
bpo-39264 to fix UserDict.get().
bpo-39267 to fix dict.__missing__ & dict.get() documentation.
Both include a PR.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IQZ4S6BBIIKIOV4NTZ633SDFZC3D5M6J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: new syntax

[Bar Harel]

This idea is proposed at least once a year. You're able to read past
discussions on python-ideas for the full list of rejection reasons.

On Mon, Mar 2, 2020, 11:28 PM Mariatta  wrote:

> -1 from me.
>
> Seems unintuitive and I don't think it's all that obvious what
> intput/floatput would mean without first reading the docs. It adds
> maintenance burden while accomplishing little value.
>
> You might be able to just create your own third party library though.
>
>
>
> On Mon, Mar 2, 2020, 1:11 PM Luca Wolf  wrote:
>
>> Dear Python Team,
>>
>> I have a good idea for a new comand in Python.
>>
>> It’s „intput()“ for numbers. So you don’t have to write „int(Input())“.
>>
>> It’s much easier to write and it’s faster too.
>>
>> You can use the same for „floatput()“, so don’t have to write
>> „float(Input())“.
>>
>>
>>
>> I hope you can make something good with this idea.
>>
>> LG: Wolf Luca
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/V5GKRDK2J2DJCRR5RIZ4756PVPH2LKRR/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HON274UA7YFLV5KGZ7RPO6CHYS2JJ5OI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/K5F6H55WHRGH6BLO3VBOGC45WAJCJLTU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How to respond to repeated bad ideas

[Bar Harel]

I wrote it and take full responsibility, did not mean to be disrespectful.
Afterall, it's not what any of us are here for.

For repeated suggestions though, that were rejected once or more in the
past, what are the general guidelines of answering?
In this case, apart from the rejection reason, we wish to teach the newbie
how to check if an idea was already suggested, else his ideas would just be
rejected over and over again.

Atm we don't have an index of ideas, apart from pep 3099, and I'm not sure
we can make one (can we?), so I do not see a way to prevent this from
happening.

I bet that many ideas are repeated, we don't know it, and it therefore
causes a repeated discussion with time and effort going nowhere.

Just to clarify, my aim by answering it this way was to reduce the overall
effort of both sides, and the attempt to reinvent the wheel. It's much like
how duplicate stackoverflow questions are closed.

On Mon, Mar 2, 2020, 11:58 PM Guido van Rossum  wrote:

> Somebody wrote, in response to a newbie's idea:
>
>> This idea is proposed at least once a year. You're able to read past
>> discussions on python-ideas for the full list of rejection reasons.
>>
>
> This   needs to go. It's unfair to the newbie, and I bet that in most
> cases whoever posts this doesn't have the slightest idea on how to find the
> specific rejection reason for the idea to which they're responding. I agree
> that many bad ideas come up repeatedly. But unless the proposer is a
> regular harasser of one of the lists, a response like this is unhelpful.
> Even if an idea is bad, let's respond in a friendly way, so the OP actually
> learns something about how Python is designed, or how it works, or whatever
> is relevant to understanding why their idea won't fly.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EZMCYMLYJKOIUWXIHUHQTQ2776XZOZ3Z/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DZ2KG2QXOEK66JTN23OZHV3HXEUDY4B2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: How to respond to repeated bad ideas

[Bar Harel]

What I usually do btw is just search on mailman. Perhaps we can guide
people to search on mailman before suggesting an idea?

On Tue, Mar 3, 2020 at 12:30 AM Bar Harel  wrote:

> I wrote it and take full responsibility, did not mean to be disrespectful.
> Afterall, it's not what any of us are here for.
>
> For repeated suggestions though, that were rejected once or more in the
> past, what are the general guidelines of answering?
> In this case, apart from the rejection reason, we wish to teach the newbie
> how to check if an idea was already suggested, else his ideas would just be
> rejected over and over again.
>
> Atm we don't have an index of ideas, apart from pep 3099, and I'm not sure
> we can make one (can we?), so I do not see a way to prevent this from
> happening.
>
> I bet that many ideas are repeated, we don't know it, and it therefore
> causes a repeated discussion with time and effort going nowhere.
>
> Just to clarify, my aim by answering it this way was to reduce the overall
> effort of both sides, and the attempt to reinvent the wheel. It's much like
> how duplicate stackoverflow questions are closed.
>
> On Mon, Mar 2, 2020, 11:58 PM Guido van Rossum  wrote:
>
>> Somebody wrote, in response to a newbie's idea:
>>
>>> This idea is proposed at least once a year. You're able to read past
>>> discussions on python-ideas for the full list of rejection reasons.
>>>
>>
>> This   needs to go. It's unfair to the newbie, and I bet that in most
>> cases whoever posts this doesn't have the slightest idea on how to find the
>> specific rejection reason for the idea to which they're responding. I agree
>> that many bad ideas come up repeatedly. But unless the proposer is a
>> regular harasser of one of the lists, a response like this is unhelpful.
>> Even if an idea is bad, let's respond in a friendly way, so the OP actually
>> learns something about how Python is designed, or how it works, or whatever
>> is relevant to understanding why their idea won't fly.
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>> <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/EZMCYMLYJKOIUWXIHUHQTQ2776XZOZ3Z/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3YYFGF2CVYJXM6PHZETWQUJ22SOUYIKM/
Code of Conduct: http://python.org/psf/codeofconduct/