[Python-Dev] Bytcode "magic tag"
In issue 25483 I'm adding an opcode to make f-string formatting more
robust and faster. As part of that, I'm bumping the .pyc magic number.
While doing that, I notice Lib/importlib/_bootstrap_external.h includes
this comment:
# Starting with the adoption of PEP 3147 in Python 3.2, every bump in magic
# number also includes a new "magic tag", i.e. a human readable string used
# to represent the magic number in __pycache__ directories. When you change
# the magic number, you must also set a new unique magic tag. Generally
this
# can be named after the Python major version of the magic number bump, but
# it can really be anything, as long as it's different than anything else
# that's come before. The tags are included in the following table,
starting
# with Python 3.2a0.
The "following table" is a comment, that contains a few references to
the tag "cpython-", specifically cpython-32. It doesn't seem
that the tag is routinely updated in the comment.
sys.implementation.cache_tag returns 'cpython-36', and is in fact
implemented as 'cpython-{PY_MAJOR_VERSION}{PY_MINOR_VERSION}'.
Do I need to do anything else? Unlike what the comment in
_boostrap_external.py suggests, this "magic tag" will not change every
time a bytecode is added, but only on every minor release. Which implies
that if we have a micro release that adds an opcode, we'll in fact break
the promise in the comment.
>From my understanding on how this tag is used, this wouldn't be a
problem (because the magic number in the file also changes). But I want
to make sure I'm not misunderstanding something. I think the comment
above is probably just misleading.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytcode "magic tag"
On Oct 28, 2015, at 08:35 AM, Eric V. Smith wrote:
>The "following table" is a comment, that contains a few references to
>the tag "cpython-", specifically cpython-32. It doesn't seem
>that the tag is routinely updated in the comment.
IIRC, it used to have to be changed in the code, but with this...
>sys.implementation.cache_tag returns 'cpython-36', and is in fact
>implemented as 'cpython-{PY_MAJOR_VERSION}{PY_MINOR_VERSION}'.
...I don't believe it does any more.
>Do I need to do anything else?
I don't think so.
>Unlike what the comment in _boostrap_external.py suggests, this "magic tag"
>will not change every time a bytecode is added, but only on every minor
>release. Which implies that if we have a micro release that adds an opcode,
>we'll in fact break the promise in the comment.
Right. Have we ever done that though? We shouldn't!
>From my understanding on how this tag is used, this wouldn't be a
>problem (because the magic number in the file also changes). But I want
>to make sure I'm not misunderstanding something. I think the comment
>above is probably just misleading.
Yeah, it's just out of date.
Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytcode "magic tag"
On 10/28/2015 10:19 AM, Barry Warsaw wrote:
> On Oct 28, 2015, at 08:35 AM, Eric V. Smith wrote:
>
>> The "following table" is a comment, that contains a few references to
>> the tag "cpython-", specifically cpython-32. It doesn't seem
>> that the tag is routinely updated in the comment.
>
> IIRC, it used to have to be changed in the code, but with this...
>
>> sys.implementation.cache_tag returns 'cpython-36', and is in fact
>> implemented as 'cpython-{PY_MAJOR_VERSION}{PY_MINOR_VERSION}'.
>
> ...I don't believe it does any more.
Okay. Maybe I'll update that comment, then.
>> Unlike what the comment in _boostrap_external.py suggests, this "magic tag"
>> will not change every time a bytecode is added, but only on every minor
>> release. Which implies that if we have a micro release that adds an opcode,
>> we'll in fact break the promise in the comment.
>
> Right. Have we ever done that though? We shouldn't!
And maybe I'll add that to the updated comment!
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytcode "magic tag"
On 10/28/2015 10:22 AM, Eric Snow wrote: > On Wed, Oct 28, 2015 at 6:35 AM, Eric V. Smith wrote: >> Do I need to do anything else? Unlike what the comment in >> _boostrap_external.py suggests, this "magic tag" will not change every >> time a bytecode is added, but only on every minor release. Which implies >> that if we have a micro release that adds an opcode, we'll in fact break >> the promise in the comment. > > You will want to bump the number on the following line: > > https://hg.python.org/cpython/file/default/Lib/importlib/_bootstrap_external.py#l231 Thanks. That part I've done (but forgot to mention). I was just concerned about the "magic tag" part, which Barry cleared up. Eric. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytcode "magic tag"
On Wed, Oct 28, 2015 at 6:35 AM, Eric V. Smith wrote:
> In issue 25483 I'm adding an opcode to make f-string formatting more
> robust and faster. As part of that, I'm bumping the .pyc magic number.
>
> While doing that, I notice Lib/importlib/_bootstrap_external.h includes
> this comment:
>
> # Starting with the adoption of PEP 3147 in Python 3.2, every bump in magic
> # number also includes a new "magic tag", i.e. a human readable string used
> # to represent the magic number in __pycache__ directories. When you change
> # the magic number, you must also set a new unique magic tag. Generally
> this
> # can be named after the Python major version of the magic number bump, but
> # it can really be anything, as long as it's different than anything else
> # that's come before. The tags are included in the following table,
> starting
> # with Python 3.2a0.
>
> The "following table" is a comment, that contains a few references to
> the tag "cpython-", specifically cpython-32. It doesn't seem
> that the tag is routinely updated in the comment.
>
> sys.implementation.cache_tag returns 'cpython-36', and is in fact
> implemented as 'cpython-{PY_MAJOR_VERSION}{PY_MINOR_VERSION}'.
>
> Do I need to do anything else? Unlike what the comment in
> _boostrap_external.py suggests, this "magic tag" will not change every
> time a bytecode is added, but only on every minor release. Which implies
> that if we have a micro release that adds an opcode, we'll in fact break
> the promise in the comment.
You will want to bump the number on the following line:
https://hg.python.org/cpython/file/default/Lib/importlib/_bootstrap_external.py#l231
-eric
>
> From my understanding on how this tag is used, this wouldn't be a
> problem (because the magic number in the file also changes). But I want
> to make sure I'm not misunderstanding something. I think the comment
> above is probably just misleading.
>
> Eric.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Intended Usage of collections.abc for Custom Collections
Dear Python-Dev, I am the author of bidict, a bidirectional map implementation for Python. A user recently filed a bug that bidict should be a subclass of dict, so that isinstance(mybidict, dict) would return True. I replied that the user should instead use isinstance(mybidict, collections.abc.Mapping), which does already return True, and is more polymorphic to boot. But before I put the issue to bed, I want to make sure I'm correctly understanding the intended usage of collections.abc, as well as any relevant interfaces I'm not currently using (collections.UserDict? __subclasshook__?), since the documentation leaves me with some doubt. Could any collections experts on this list please confirm whether bidict is implemented as the language intends it should be? Some quick references: https://bidict.readthedocs.org/en/latest/other-bidict-types.html#bidict-type-hierarchy https://github.com/jab/bidict/blob/master/bidict/_bidict.py I would be happy to try to capture what I learn from this thread and write up a guide for collections library authors in the future, or otherwise pay your help forward however I can. Thanks and best wishes. -jab ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bytcode "magic tag"
On Wed, Oct 28, 2015 at 8:28 AM, Eric V. Smith wrote: > Thanks. That part I've done (but forgot to mention). I was just > concerned about the "magic tag" part, which Barry cleared up. Ah, I misread. :) Yeah, that comment is out of date. -eric ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intended Usage of collections.abc for Custom Collections
On Wed, 28 Oct 2015 at 08:47 wrote: > Dear Python-Dev, > > I am the author of bidict, a bidirectional map implementation for Python. > A user recently filed a bug that bidict should be a subclass of dict, so > that isinstance(mybidict, dict) would return True. I replied that the user > should instead use isinstance(mybidict, collections.abc.Mapping), which > does already return True, and is more polymorphic to boot. > I would argue that chances are they don't need isinstance() in either case. :) > > But before I put the issue to bed, I want to make sure I'm correctly > understanding the intended usage of collections.abc, as well as any > relevant interfaces I'm not currently using (collections.UserDict? > __subclasshook__?), since the documentation leaves me with some doubt. > Could any collections experts on this list please confirm whether bidict is > implemented as the language intends it should be? > ABCs are meant to make sure you implement key methods for an interface/protocol. So in the case of collections.abc.Mapping, it's to make sure you implement __getitem__. In exchange for subclassing the ABC you also gain some methods for free like get(). So you subclass an ABC because you want your object to be acceptable in any code that expects an object that implements that interface/protocol and you want the help ABCs provide in making sure you don't accidentally miss some key method. Subclassing a concrete implementation of the Mapping ABC -- which is what dict is -- should be done if it is beneficial to you, but not simply to satisfy an isinstance() check. I think the ABC registration is the right thing to do and the user requesting the dict subclass should actually be doing what you suggested and testing for the interface/protocol and not the concrete implementation. And if you want another way to hit this point home, with type hints people should only be expecting abstract types like typing.Mapping as input: https://docs.python.org/3/library/typing.html#typing.Mapping . Restricting yourself to only a dict locks out other completely viable types that implement the mapping interface/protocol. Much like working with data, you should be as flexible as possible on your inputs (e.g., specifying typing.Mapping as the parameter type), but as strict as possible on the return type (.e.g, specifying dict/typing.Dict as the return type). I honestly would want to know why the user cares about an isinstance() check to begin with since they might want to go with a try/except when using the object how they want it to be and erroring out if they get passed an object that doesn't quack like a dict thanks to duck typing. -Brett > > Some quick references: > > https://bidict.readthedocs.org/en/latest/other-bidict-types.html#bidict-type-hierarchy > https://github.com/jab/bidict/blob/master/bidict/_bidict.py > > I would be happy to try to capture what I learn from this thread and write > up a guide for collections library authors in the future, or otherwise pay > your help forward however I can. > > Thanks and best wishes. > > -jab > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Intended Usage of collections.abc for Custom Collections
On Wed, Oct 28, 2015 at 1:16 PM, Brett Cannon wrote: > On Wed, 28 Oct 2015 at 08:47 wrote: > >> Dear Python-Dev, >> >> I am the author of bidict, a bidirectional map implementation for Python. >> A user recently filed a bug that bidict should be a subclass of dict, so >> that isinstance(mybidict, dict) would return True. I replied that the user >> should instead use isinstance(mybidict, collections.abc.Mapping), which >> does already return True, and is more polymorphic to boot. >> > > I would argue that chances are they don't need isinstance() in either > case. :) > > >> But before I put the issue to bed, I want to make sure I'm correctly >> understanding the intended usage of collections.abc, as well as any >> relevant interfaces I'm not currently using (collections.UserDict? >> __subclasshook__?), since the documentation leaves me with some doubt. >> Could any collections experts on this list please confirm whether bidict is >> implemented as the language intends it should be? >> > > ABCs are meant to make sure you implement key methods for an > interface/protocol. So in the case of collections.abc.Mapping, it's to make > sure you implement __getitem__. In exchange for subclassing the ABC you > also gain some methods for free like get(). So you subclass an ABC because > you want your object to be acceptable in any code that expects an object > that implements that interface/protocol and you want the help ABCs provide > in making sure you don't accidentally miss some key method. > > Subclassing a concrete implementation of the Mapping ABC -- which is what > dict is -- should be done if it is beneficial to you, but not simply to > satisfy an isinstance() check. I think the ABC registration is the right > thing to do and the user requesting the dict subclass should actually be > doing what you suggested and testing for the interface/protocol and not the > concrete implementation. > > And if you want another way to hit this point home, with type hints people > should only be expecting abstract types like typing.Mapping as input: > https://docs.python.org/3/library/typing.html#typing.Mapping . > Restricting yourself to only a dict locks out other completely viable types > that implement the mapping interface/protocol. Much like working with data, > you should be as flexible as possible on your inputs (e.g., specifying > typing.Mapping as the parameter type), but as strict as possible on the > return type (.e.g, specifying dict/typing.Dict as the return type). > > I honestly would want to know why the user cares about an isinstance() > check to begin with since they might want to go with a try/except when > using the object how they want it to be and erroring out if they get passed > an object that doesn't quack like a dict thanks to duck typing. > > -Brett > Thanks very much for the thorough and thoughtful reply. I'll take this as authoritative approval of the current design, barring any further recommendations to the contrary. As for the isinstance check, it turned out that this wasn't actually in the user's code; the offending code is actually in the pandas library, which he was using. I just submitted a PR there in case anyone is interested: https://github.com/pydata/pandas/pull/11461 Thanks again for making my first experience on this list so positive. -jab ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should PEP 498 specify if rf'...' is valid?
Have you ever used a command line application that --accepted --Boolean --flags? Have you ever found one that required the flags to be in order? You remember how much you hated that application for being so arbitrary about the input? That is exactly how I feel about the order mattering for string prefixes in 2.7. 3.x got rid of that, and that is one of the few undeniably good choices made in python 3. for the love of all that is holy, don't un-fix that. On 10/27/2015 14:39, Sven R. Kunze wrote: On 26.10.2015 20:54, Ethan Furman wrote: You misunderstand -- the order of string prefixes does *not* matter, so forcing us to use one is silly. I don't think so. It's better to have less possibilities in the beginning. So later on, we can easily relax the restrictions without breaking compatibility. Best, Sven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-list%40sdamon.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
