[Python-Dev] Fwd: Python programming language vulnerabilities
This popped up on python-list. It actually seems to me like it might be interesting to the core developers. Apologies if I've missed my guess. Skip -- Forwarded message -- From: Stephen Michell Date: Fri, Sep 8, 2017 at 12:34 PM Subject: Python programming language vulnerabilities To: python-l...@python.org I chair ISO/IEC/JTC1/SC22/WG23 Programming Language Vulnerabilities. We publish an international technical report, ISO IEC TR 24772 Guide to avoiding programming language vulnerabilities through language selection use. Annex D in this document addresses vulnerabilities in Python. This document is freely available from ISO and IEC. We are updating this technical report, adding a few vulnerabilities and updating language applicability as programming languages evolve. We are also subdividing the document by making the language-specific annexes each their own technical report. For the Python Part, the major portions are written, but we have about 6 potential vulnerabilities left to complete. We need help in finishing the Python TR. We are looking for a few Python experts that have experience in implementing Python language systems, or experts in implementing significant systems in Python (for technical level, persons that provide technical supervision to implementers, or that write and maintain organizational Python coding standards. If you are interested in helping, please reply to this posting. Thank you Stephen Michell Convenor, ISO/IEC/JTC 1/SC 22/WG 23 Programming Language Vulnerabilities -- https://mail.python.org/mailman/listinfo/python-list ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
Don't we already have the mock module for that ? A mowk works as a noop, will be ok with being used as a context manager and allow chaining... Either way, what would a noop function really give you compared to lambda *a, **b: None ? A be bit shorter to write. Maybe faster to run. But do you use it so much that it needs to be included ? It's not rocket science. And as a built-in, noon of the less. If it's coded it in C to gain perfs, then: - let's put it in functools, not in built-in. I often wish for partial to be built-in, but it's not. Honestly the fonctools and itertools module should be autoimported (I always do in my PYTHONSTARTUP). But no pony for us, let's not clutter the global name spaces. - provide it with it's little sister, the identity function. They almost always go hand in hand and doing the whole work of this PEP is a nice opportunity. sorted/max/min/sort and most validation callbacks have an identity function as default parameters. We would have then functools.noop and functools.identity. - provide a pure Python backport. Aternatively, just rewrite part of the mock module in C. You'll get a fast noop, with a lot of features, and as a bonus would speed up a lot of unit tests around here. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
The reaction is overwhelmingly positive everywhere: hacker news, reddit, twitter. People have been expecting something like that for a long time. 3 questions: - is providing validation/conversion hooks completely out of the question of still open for debate ? I know it's to keep the implementation simple but have a few callbacks run in the __init__ in a foo loop is not that much complexity. You don't have to provide validators, but having a validators parameters on field() would be a huge time saver. Just a list of callables called when the value is first set, potentially raising an exception that you don't even need to process in any way. It returns the value converted, and voilà. We all do that every day manually. - I read Guido talking about some base class as alternative to the generator version, but don't see it in the PEP. Is it still considered ? - any chance it becomes a built in later ? When classes have been improved in Python 2, the object built-in was added. Imagine if we had had to import it every time... Or maybe just plug it to object like @object.dataclass. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 9/10/2017 10:00 AM, Michel Desmoulin wrote:
The reaction is overwhelmingly positive everywhere: hacker news, reddit,
twitter.
Do you have a pointer to the Hacker News discussion? I missed it.
People have been expecting something like that for a long time.
Me, too!
3 questions:
- is providing validation/conversion hooks completely out of the
question of still open for debate ? I know it's to keep the
implementation simple but have a few callbacks run in the __init__ in a
foo loop is not that much complexity. You don't have to provide
validators, but having a validators parameters on field() would be a
huge time saver. Just a list of callables called when the value is first
set, potentially raising an exception that you don't even need to
process in any way. It returns the value converted, and voilà. We all do
that every day manually.
I don't particularly want to add validation specifically. I want to make
it possible to add validation yourself, or via a library.
What I think I'll do is add a metadata parameter to fields(), defaulting
to None. Then you could write a post-init hook that does whatever
single- and multi-field validations you want (or whatever else you want
to do). Although this plays poorly with "frozen" classes: it's always
something! I'll think about it.
To make this most useful, I need to get the post-init hook to take an
optional parameter so you can get data to it. I don't have a good way to
do this, yet. Suggestions welcomed.
Although if the post-init hook takes a param that you can pass in at
object creation time, I guess there's really no need for a per-field
metadata parameter: you could use the field name as a key to look up
whatever you wanted to know about the field.
- I read Guido talking about some base class as alternative to the
generator version, but don't see it in the PEP. Is it still considered ?
I'm going to put some words in explaining why I don't want to use base
classes (I don't think it buys you anything). Do you have a reason for
preferring base classes?
- any chance it becomes a built in later ? When classes have been
improved in Python 2, the object built-in was added. Imagine if we had
had to import it every time... Or maybe just plug it to object like
@object.dataclass.
Because of the choice of using module-level functions so as to not
introduce conflicts in the object's namespace, it would be difficult to
make this a builtin.
Although now that I think about it, maybe what are currently
module-level functions should instead be methods on the "dataclass"
decorator itself:
@dataclass
class C:
i: int = dataclass.field(default=1, init=False)
j: str
c = C('hello')
dataclass.asdict(c)
{'i': 1, 'j': 'hello'}
Then, "dataclass" would be the only name the module exports, making it
easier to someday be a builtin. I'm not sure it's important enough for
this to be a builtin, but this would make it easier. Thoughts? I'm
usually not a fan of having attributes on a function: it's why
itertools.chain.from_iterable() is hard to find.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
On Sep 9, 2017, at 15:12, Guido van Rossum wrote: > > I can't tell whether this was meant seriously, but I don't think it's worth > it. People can easily write their own dummy function and give it any damn > semantics they want. Let's reject the PEP. Alrighty then! (Yes, it was serious, but I claim post-sprint euphoria/delirium). -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On Sun, Sep 10, 2017 at 9:36 AM, Eric V. Smith wrote:
> On 9/10/2017 10:00 AM, Michel Desmoulin wrote:
>
>> - any chance it becomes a built in later ? When classes have been
>> improved in Python 2, the object built-in was added. Imagine if we had
>> had to import it every time... Or maybe just plug it to object like
>> @object.dataclass.
>>
>
> Because of the choice of using module-level functions so as to not
> introduce conflicts in the object's namespace, it would be difficult to
> make this a builtin.
>
The temptation to make everything a builtin should be resisted.
> Although now that I think about it, maybe what are currently module-level
> functions should instead be methods on the "dataclass" decorator itself:
>
> @dataclass
> class C:
> i: int = dataclass.field(default=1, init=False)
> j: str
>
> c = C('hello')
>
> dataclass.asdict(c)
> {'i': 1, 'j': 'hello'}
>
> Then, "dataclass" would be the only name the module exports, making it
> easier to someday be a builtin. I'm not sure it's important enough for this
> to be a builtin, but this would make it easier. Thoughts? I'm usually not a
> fan of having attributes on a function: it's why
> itertools.chain.from_iterable() is hard to find.
>
Let's not do that.
It would be better to design the module so that people can write `from
dataclasses import *` and they will only get things that are clearly part
of dataclasses (I guess dataclass, field, asdict, and a few more like
that). That way people who really want this to look like a builtin can just
violate PEP 8.
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] breakpoint() and $PYTHONBREAKPOINT
For PEP 553, I think it’s a good idea to support the environment variable $PYTHONBREAKPOINT[*] but I’m stuck on a design question, so I’d like to get some feedback. Should $PYTHONBREAKPOINT be consulted in breakpoint() or in sys.breakpointhook()? If we support it in breakpoint() then it means if $PYTHONBREAKPOINT is set, Python will never call sys.breakpointhook() so programmatic override of that will be ignored. If we support it in sys.breakpointhook(), then if you programmatically override that, $PYTHONBREAKPOINT will be ignored even if set. Of course, any code that overrides sys.breakpointhook() could also consult $PYTHONBREAKPOINT if they want. So how do we want it to work? It seems like explicit code should override the environment variable, which should override the default, meaning that $PYTHONBREAKPOINT should be consulted only in breakpoint(). And if you set sys.breakpointhook() with something else, then oh well, Python just ignores $PYTHONBREAKPOINT. Feedback welcome. -Barry [*] Probably not $PYTHONBREAKPOINTHOOK although if we choose to implement that in sys.breakpointhook(), it might still makes sense. signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] breakpoint() and $PYTHONBREAKPOINT
I think programmatic overrides should be able to decide for themselves if they want to honor PYTHONBREAKPOINT or not, since I can imagine use cases that go both ways. So it should be checked in sys.breakpointhook(). On Sun, Sep 10, 2017 at 12:06 PM, Barry Warsaw wrote: > For PEP 553, I think it’s a good idea to support the environment variable > $PYTHONBREAKPOINT[*] but I’m stuck on a design question, so I’d like to get > some feedback. > > Should $PYTHONBREAKPOINT be consulted in breakpoint() or in > sys.breakpointhook()? > > If we support it in breakpoint() then it means if $PYTHONBREAKPOINT is > set, Python will never call sys.breakpointhook() so programmatic override > of that will be ignored. > > If we support it in sys.breakpointhook(), then if you programmatically > override that, $PYTHONBREAKPOINT will be ignored even if set. Of course, > any code that overrides sys.breakpointhook() could also consult > $PYTHONBREAKPOINT if they want. > > So how do we want it to work? It seems like explicit code should override > the environment variable, which should override the default, meaning that > $PYTHONBREAKPOINT should be consulted only in breakpoint(). And if you set > sys.breakpointhook() with something else, then oh well, Python just ignores > $PYTHONBREAKPOINT. > > Feedback welcome. > -Barry > > [*] Probably not $PYTHONBREAKPOINTHOOK although if we choose to implement > that in sys.breakpointhook(), it might still makes sense. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] breakpoint() and $PYTHONBREAKPOINT
On Sun, Sep 10, 2017 at 12:06 PM, Barry Warsaw wrote: > For PEP 553, I think it’s a good idea to support the environment variable > $PYTHONBREAKPOINT[*] but I’m stuck on a design question, so I’d like to get > some feedback. > > Should $PYTHONBREAKPOINT be consulted in breakpoint() or in > sys.breakpointhook()? Wouldn't the usual pattern be to check $PYTHONBREAKPOINT once at startup, and if it's set use it to initialize sys.breakpointhook()? Compare to, say, $PYTHONPATH. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Thanks for the info. On 2017-09-08 15:40, Eric V. Smith wrote: - Are types required? Annotations are required, the typing module is not. Maybe an example or two with Any? I'd rather leave it like it is: typing is referenced only once, for ClassVar. Guess I really meant "object" or "type" above, not "typing.Any." For a decade or two, my use of structs/records in Python has been dynamically-typed and that hasn't been an issue. As the problem this PEP is solving is orthogonal to typing improvements, it feels a bit like typing is being coupled and pushed with it, whether wanted or not. Not that I dislike typing mind you (appreciated on projects with large teams), it's just as mentioned not related to the problem of class definition verbosity or lacking functionality. Cheers, -Mike ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Thanks for the info. On 2017-09-08 15:40, Eric V. Smith wrote: - Are types required? Annotations are required, the typing module is not. Maybe an example or two with Any? I'd rather leave it like it is: typing is referenced only once, for ClassVar. Guess I really meant "object" or "type" above, not "typing.Any." For a decade or two, my use of structs/records in Python has been dynamically-typed and that hasn't been an issue. As the problem this PEP is solving is orthogonal to typing improvements, it feels a bit like typing is being coupled and pushed with it, whether wanted or not. Not that I dislike typing mind you (appreciated on projects with large teams), it's just as mentioned not related to the problem of class definition verbosity or lacking functionality. Cheers, -Mike ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 10 September 2017 at 23:05, Mike Miller wrote: > [...] > As the problem this PEP is solving is orthogonal to typing improvements > This is not the case, static support for dataclasses is an import point of motivation. It is hard to support static typing for many third party packages like attrs, since they use a lot of "magic". -- Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
On Sun, Sep 10, 2017 at 8:21 PM, Barry Warsaw wrote: > On Sep 9, 2017, at 15:12, Guido van Rossum wrote: > > > > I can't tell whether this was meant seriously, but I don't think it's > worth it. People can easily write their own dummy function and give it any > damn semantics they want. Let's reject the PEP. > > Alrighty then! (Yes, it was serious, but I claim post-sprint > euphoria/delirium). > > Just for future reference, here's a slightly more serious comment: I think the "pass" statement wasn't mentioned yet, but clearly noop() would be duplication of functionality. So maybe the closest thing without duplication would be to make "pass" an expression which evaluates to a no-op function, but which the compiler could perhaps optimize away if it's a statement by itself, or is a builtin. -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven + ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
On Mon, Sep 11, 2017 at 7:29 AM, Koos Zevenhoven wrote: > On Sun, Sep 10, 2017 at 8:21 PM, Barry Warsaw wrote: >> >> On Sep 9, 2017, at 15:12, Guido van Rossum wrote: >> > >> > I can't tell whether this was meant seriously, but I don't think it's >> > worth it. People can easily write their own dummy function and give it any >> > damn semantics they want. Let's reject the PEP. >> >> Alrighty then! (Yes, it was serious, but I claim post-sprint >> euphoria/delirium). >> > > Just for future reference, here's a slightly more serious comment: > > I think the "pass" statement wasn't mentioned yet, but clearly noop() would > be duplication of functionality. So maybe the closest thing without > duplication would be to make "pass" an expression which evaluates to a no-op > function, but which the compiler could perhaps optimize away if it's a > statement by itself, or is a builtin. As a language change, definitely not. But I like this idea for PYTHONBREAKPOINT. You set it to the name of a function, or to "pass" if you want nothing to be done. It's a special case that can't possibly conflict with normal usage. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 2017-09-10 14:23, Ivan Levkivskyi wrote: This is not the case, static support for dataclasses is an import point of motivation. I've needed this functionality a decade before types became cool again. ;-) It is hard to support static typing for many third party packages like attrs, since they use a lot of "magic". Believe that attrs has its own validators. As mentioned, nothing against static typing, would simply like an example without, to show that it is not required. -Mike ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
On 9/8/2017 11:01 AM, Eric V. Smith wrote: Oops, I forgot the link. It should show up shortly at https://www.python.org/dev/peps/pep-0557/. And now I've pushed a version that works with Python 3.6 to PyPI at https://pypi.python.org/pypi/dataclasses It implements the PEP as it currently stands. I'll be making some tweaks in the coming weeks. Feedback is welcomed. The repo is at https://github.com/ericvsmith/dataclasses Eric. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
On Sep 10, 2017, at 14:39, Chris Angelico wrote: > > As a language change, definitely not. But I like this idea for > PYTHONBREAKPOINT. You set it to the name of a function, or to "pass" > if you want nothing to be done. It's a special case that can't > possibly conflict with normal usage. I have working code for `PYTHONBREAKPOINT=0` and `PYTHONBREAKPOINT= ` (i.e. the empty string as given by getenv()) meaning “disable”. I don’t think we also need `PYTHONBREAKPOINT=pass`. Cheers, -Barry signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
On Mon, Sep 11, 2017 at 11:03 AM, Barry Warsaw wrote: > On Sep 10, 2017, at 14:39, Chris Angelico wrote: >> >> As a language change, definitely not. But I like this idea for >> PYTHONBREAKPOINT. You set it to the name of a function, or to "pass" >> if you want nothing to be done. It's a special case that can't >> possibly conflict with normal usage. > > I have working code for `PYTHONBREAKPOINT=0` and `PYTHONBREAKPOINT= ` (i.e. > the empty string as given by getenv()) meaning “disable”. I don’t think we > also need `PYTHONBREAKPOINT=pass`. > Blank is the other one that I would stumble to. Works for me. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
> On Sep 10, 2017, at 5:13 PM, Mike Miller wrote: > > Thanks for the info. > > On 2017-09-08 15:40, Eric V. Smith wrote: >>> - Are types required? >> Annotations are required, the typing module is not. >>> Maybe an example or two with Any? >> I'd rather leave it like it is: typing is referenced only once, for ClassVar. > > > Guess I really meant "object" or "type" above, not "typing.Any." > > For a decade or two, my use of structs/records in Python has been > dynamically-typed and that hasn't been an issue. As the problem this PEP is > solving is orthogonal to typing improvements, it feels a bit like typing is > being coupled and pushed with it, whether wanted or not. > > Not that I dislike typing mind you (appreciated on projects with large > teams), it's just as mentioned not related to the problem of class definition > verbosity or lacking functionality. Using type annotations buys two things. One is the concise syntax: it's how the decorator finds the fields. In the simple case, it's what let's you define a data class without using the call to fields(), which is analogous to attrs's attr.ib(). The other thing it buys you is compatibility with type checkers. As Ivan said, the design was very careful to be compatible with type checkers. So other than requiring some type as an annotation, there's no dependency added for typing in the genera sense. You can use a type of object if you want, or just lie to it and say None (though I'm not recommending that!). It's completely ignored at runtime (except for the ClassVar case, as the PEP states). I'll think about adding some more language to the PEP about it. Eric. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 559 - built-in noop()
On Mon, Sep 11, 2017 at 07:39:07AM +1000, Chris Angelico wrote: [...] > As a language change, definitely not. But I like this idea for > PYTHONBREAKPOINT. You set it to the name of a function, or to "pass" > if you want nothing to be done. It's a special case that can't > possibly conflict with normal usage. I disagree -- its a confusion of concepts. "pass" is a do-nothing statement, not a value, so you can't set something to pass. Expect a lot of StackOverflow questions asking why this doesn't work: sys.breakpoint = pass In fact, in one sense pass is not even a statement. It has no runtime effect, it isn't compiled into any bytecode. It is a purely syntactic feature to satisfy the parser. Of course env variables are actually strings, so we can choose "pass" to mean "no break point" if we wanted. But I think there are already two perfectly good candidates for that usage which don't mix the concepts of statements and values, the empty string, and None: setenv PYTHONBREAKPOINT="" setenv PYTHONBREAKPOINT=None -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
> On Sep 10, 2017, at 4:54 PM, Eric V. Smith wrote: > > And now I've pushed a version that works with Python 3.6 to PyPI at > https://pypi.python.org/pypi/dataclasses > > It implements the PEP as it currently stands. I'll be making some tweaks in > the coming weeks. Feedback is welcomed. > > The repo is at https://github.com/ericvsmith/dataclasses +1 Overall, this looks very well thought out. Nice work! Once you get agreement on the functionality, name bike-shedding will likely be next. In a way, all classes are data classes so that name doesn't tell me much. Instead, it would be nice to have something suggestive of what it actually does which is automatically adding boilerplate methods to a general purpose class. Perhaps, @boilerplate or @autoinit or some such. Raymond ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Hi Eric,
A few quick comments:
Why do you even have a hash= argument on individual fields? For the whole
class, I can imagine you might want to explicitly mark a whole class as
unhashable, but it seems like the only thing you can do with the
field-level hash= argument is to create a class where the __hash__ and
__eq__ take different fields into account, and why would you ever want that?
Though honestly I can see a reasonable argument for removing the
class-level hash= option too. And even if you keep it you might want to
error on some truly nonsensical options like defining __hash__ without
__eq__. (Also watch out that Python's usual rule about defining __eq__
blocking the inheritance of __hash__ does not kick in if __eq__ is added
after the class is created.)
I've sometimes wished that attrs let me control whether it generated
equality methods (eq/ne/hash) separately from ordering methods (lt/gt/...).
Maybe the cmp= argument should take an enum with options
none/equality-only/full?
The "why not attrs" section kind of reads like "because it's too popular
and useful"?
-n
On Sep 8, 2017 08:44, "Eric V. Smith" wrote:
Oops, I forgot the link. It should show up shortly at
https://www.python.org/dev/peps/pep-0557/.
Eric.
On 9/8/17 7:57 AM, Eric V. Smith wrote:
> I've written a PEP for what might be thought of as "mutable namedtuples
> with defaults, but not inheriting tuple's behavior" (a mouthful, but it
> sounded simpler when I first thought of it). It's heavily influenced by
> the attrs project. It uses PEP 526 type annotations to define fields.
> From the overview section:
>
> @dataclass
> class InventoryItem:
> name: str
> unit_price: float
> quantity_on_hand: int = 0
>
> def total_cost(self) -> float:
> return self.unit_price * self.quantity_on_hand
>
> Will automatically add these methods:
>
> def __init__(self, name: str, unit_price: float, quantity_on_hand: int
> = 0) -> None:
> self.name = name
> self.unit_price = unit_price
> self.quantity_on_hand = quantity_on_hand
> def __repr__(self):
> return
> f'InventoryItem(name={self.name!r},unit_price={self.unit_pri
> ce!r},quantity_on_hand={self.quantity_on_hand!r})'
>
> def __eq__(self, other):
> if other.__class__ is self.__class__:
> return (self.name, self.unit_price, self.quantity_on_hand) ==
> (other.name, other.unit_price, other.quantity_on_hand)
> return NotImplemented
> def __ne__(self, other):
> if other.__class__ is self.__class__:
> return (self.name, self.unit_price, self.quantity_on_hand) !=
> (other.name, other.unit_price, other.quantity_on_hand)
> return NotImplemented
> def __lt__(self, other):
> if other.__class__ is self.__class__:
> return (self.name, self.unit_price, self.quantity_on_hand) <
> (other.name, other.unit_price, other.quantity_on_hand)
> return NotImplemented
> def __le__(self, other):
> if other.__class__ is self.__class__:
> return (self.name, self.unit_price, self.quantity_on_hand) <=
> (other.name, other.unit_price, other.quantity_on_hand)
> return NotImplemented
> def __gt__(self, other):
> if other.__class__ is self.__class__:
> return (self.name, self.unit_price, self.quantity_on_hand) >
> (other.name, other.unit_price, other.quantity_on_hand)
> return NotImplemented
> def __ge__(self, other):
> if other.__class__ is self.__class__:
> return (self.name, self.unit_price, self.quantity_on_hand) >=
> (other.name, other.unit_price, other.quantity_on_hand)
> return NotImplemented
>
> Data Classes saves you from writing and maintaining these functions.
>
> The PEP is largely complete, but could use some filling out in places.
> Comments welcome!
>
> Eric.
>
> P.S. I wrote this PEP when I was in my happy place.
>
> ___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/njs%
40pobox.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 549: Instance Properties (aka: module properties)
On 09/06/2017 08:26 AM, Guido van Rossum wrote:
So we've seen a real use case for __class__ assignment: deprecating
things on access. That use case could also be solved if modules
natively supported defining __getattr__ (with the same "only used if
attribute not found otherwise" semantics as it has on classes), but it
couldn't be solved using @property (or at least it would be quite hacky).
I guess it's a matter of perspective. I solved this problem using
@property, and I don't think it's particularly hacky. (See my
implementation at the bottom of this email). The worst thing it does is
look up the current module via sys.modules[__name__]... which
Nathaniel's code also must do.
My example is a lot simpler than Nathaniel's code but it's just a
proof-of-concept. Nathaniel's code does much more. In particular, I
didn't override __dir__ to hide the deprecated attributes; doing that
would mean assigning to __class__ just as Nathaniel does. (If you're
actually curious to see it I could add that to the proof-of-concept.)
IMO my PEP strikes the right balance. @property is far more popular
than __getattr__ or __getattribute__; 19 times out of 20, people use
@property. Meanwhile, people for whom @property is insufficient (like
Nathaniel) can already assign to module.__class__ and override
__getattr__, __getattribute__, __del__, or any other existing magic
method. In other words: the commonplace is easy, and the unusual is
possible. Perfect!
//arry
-
/test_deprecated.py:
import depmod
print(depmod.depr1) # throws an exception
depmod.py:
# module with deprecated properties
import sys
_deprecated_properies = (
("depr1", 33),
("depr2", 44),
)
__module__ = sys.modules[__name__]
def set_deprecated_property(name, value):
@property
def prop(self):
raise RuntimeError(f"property '{name}' is deprecated")
return value
setattr(__module__, name, prop)
for name, value in _deprecated_properies:
set_deprecated_property(name, value)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 549: Instance Properties (aka: module properties)
On 09/06/2017 02:13 PM, Ronald Oussoren wrote: To be honest this sounds like a fairly crude hack. Updating the __class__ of a module object feels dirty, but at least you get normal behavior w.r.t. properties. Okay. Obviously I disagree, I think it's reasonable. But I'll assume you're -1. Why is there no mechanism to add new descriptors that can work in this context? I've updated the prototype to add one. I added it as "collections.abc.InstanceDescriptor"; that's a base class you can inherit from, and then your descriptor will work in a module. Bikeshedding the name is fine. BTW. The interaction with import is interesting… Module properties only work as naive users expect when accessing them as attributes of the module object, in particular importing the name using “from module import prop” would only call the property getter once and that may not be the intended behavior. I spent a fair amount of time thinking about this. The short answer is: we /could/ fix it. We /could/ make it so that "from x import y", when x.y is an instance descriptor, ensures that y honors the descriptor protocol when referenced. We'd have to do it for three contexts: * global scope (aka module scope) * class scope * function scope The first two are pretty similar; create a proxy object that retains the module instance so it remains "bound" to that. The third one is trickier; it'd mean a new bytecode (LOAD_FAST_DESCRIPTOR), but it wouldn't slow down people who didn't use it. Anyway, long story short, I think this would be worse than simply having "from x import y" only calling the getter once. As the Zen says: special cases aren't special enough to break the rules. //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 554 v2 (new "interpreters" module)
On 10 September 2017 at 03:54, Nathaniel Smith wrote: > On Sep 9, 2017 9:07 AM, "Nick Coghlan" wrote: > > > To immediately realise some level of efficiency benefits from the > shared memory space between the main interpreter and subinterpreters, > I also think these low level FIFOs should be defined as accepting any > object that supports the PEP 3118 buffer protocol, and emitting > memoryview() objects on the receiving end, rather than being bytes-in, > bytes-out. > > > Is your idea that this memoryview would refer directly to the sending > interpreter's memory (as opposed to a copy into some receiver-owned buffer)? There are two possibilities that I think can be made to work. Option one would be a memoryview subclass that: 1. Also stores a reference to the source interpreter 2. Temporarily switches back to that interpreter when acquiring or releasing a buffer view The receiving interpreter would then also be able to acquire a suitable referencing to the sending interpreter for the message in order to establish bidirectional communications. Option two would be the same general idea, but with a regular memoryview placed in front of the subinterpreter aware variant (although it's less clear how we'd establish bidirectional comms channels in that case). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 554 v2 (new "interpreters" module)
On 10 September 2017 at 04:04, Nathaniel Smith wrote: > On Sep 8, 2017 4:06 PM, "Eric Snow" wrote: > > >run(code): > > Run the provided Python code in the interpreter, in the current > OS thread. If the interpreter is already running then raise > RuntimeError in the interpreter that called ``run()``. > > The current interpreter (which called ``run()``) will block until > the subinterpreter finishes running the requested code. Any > uncaught exception in that code will bubble up to the current > interpreter. > > > This phrase "bubble up" here is doing a lot of work :-). Can you elaborate > on what you mean? The text now makes it seem like the exception will just > pass from one interpreter into another, but that seems impossible – it'd > mean sharing not just arbitrary user defined exception classes but full > frame objects... Indeed, I think this will need to be something more akin to https://docs.python.org/3/library/subprocess.html#subprocess.CalledProcessError, where the child interpreter is able to pass back encoded text data (perhaps including a full rendered traceback), but the exception itself won't be able to be propagated. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Python programming language vulnerabilities
On 10 September 2017 at 19:42, Skip Montanaro wrote: > This popped up on python-list. It actually seems to me like it might be > interesting to the core developers. Apologies if I've missed my guess. Thanks for passing this along. If it's the document I think it is, I've reviewed at least one previous iteration of it, so I'll let Stephen know I'll be happy to help out this time as well. (It would be highly desirable to have more than just me looking at it, though!) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 554 v2 (new "interpreters" module)
On Mon, Sep 11, 2017 at 8:51 AM, Nick Coghlan wrote: > On 10 September 2017 at 04:04, Nathaniel Smith wrote: > > On Sep 8, 2017 4:06 PM, "Eric Snow" wrote: > > > > > >run(code): > > > > Run the provided Python code in the interpreter, in the current > > OS thread. If the interpreter is already running then raise > > RuntimeError in the interpreter that called ``run()``. > > > > The current interpreter (which called ``run()``) will block until > > the subinterpreter finishes running the requested code. Any > > uncaught exception in that code will bubble up to the current > > interpreter. > > > > > > This phrase "bubble up" here is doing a lot of work :-). Can you > elaborate > > on what you mean? The text now makes it seem like the exception will just > > pass from one interpreter into another, but that seems impossible – it'd > > mean sharing not just arbitrary user defined exception classes but full > > frame objects... > > Indeed, I think this will need to be something more akin to > https://docs.python.org/3/library/subprocess.html# > subprocess.CalledProcessError, > where the child interpreter is able to pass back encoded text data > (perhaps including a full rendered traceback), but the exception > itself won't be able to be propagated. > It would be helpful if at least the exception type could somehow be preserved / restored / mimicked. Otherwise you need if-elif statements in your try-excepts and other annoying stuff. -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven + ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 557: Data Classes
Le 10/09/2017 à 18:36, Eric V. Smith a écrit :
> On 9/10/2017 10:00 AM, Michel Desmoulin wrote:
>> The reaction is overwhelmingly positive everywhere: hacker news, reddit,
>> twitter.
>
> Do you have a pointer to the Hacker News discussion? I missed it.
Err... I may have been over enthusiastic and created the hacker news
thread in my mind.
>
>> People have been expecting something like that for a long time.
>
> Me, too!
>
>> 3 questions:
>>
>> - is providing validation/conversion hooks completely out of the
>> question of still open for debate ? I know it's to keep the
>> implementation simple but have a few callbacks run in the __init__ in a
>> foo loop is not that much complexity. You don't have to provide
>> validators, but having a validators parameters on field() would be a
>> huge time saver. Just a list of callables called when the value is first
>> set, potentially raising an exception that you don't even need to
>> process in any way. It returns the value converted, and voilà. We all do
>> that every day manually.
>
> I don't particularly want to add validation specifically. I want to make
> it possible to add validation yourself, or via a library.
>
> What I think I'll do is add a metadata parameter to fields(), defaulting
> to None. Then you could write a post-init hook that does whatever
> single- and multi-field validations you want (or whatever else you want
> to do). Although this plays poorly with "frozen" classes: it's always
> something! I'll think about it.
>
> To make this most useful, I need to get the post-init hook to take an
> optional parameter so you can get data to it. I don't have a good way to
> do this, yet. Suggestions welcomed.
It doesn't really allow you to do anything you couldn't do as easily as
in __init__.
Alternatively, you could have a "on_set" hooks for field(), that just
take the field value, and return the field value. By default, it's an
identity function and is always called (minus implementation optimizations):
from functools improt reduce
self.foo = reduce((lambda data, next: next(*data)), on_set_hooks,
(field, val))
And people can do whatever they want: default values, factories,
transformers, converters/casters, validation, logging...
>
> Although if the post-init hook takes a param that you can pass in at
> object creation time, I guess there's really no need for a per-field
> metadata parameter: you could use the field name as a key to look up
> whatever you wanted to know about the field.
>
>> - I read Guido talking about some base class as alternative to the
>> generator version, but don't see it in the PEP. Is it still considered ?
>
> I'm going to put some words in explaining why I don't want to use base
> classes (I don't think it buys you anything). Do you have a reason for
> preferring base classes?
Not preferring, but having it as an alternative. Mainly for 2 reasons:
1 - data classes allow one to type in classes very quickly, let's
harvest the benefit from that.
Typing a decorator in a shell is much less comfortable than using
inheritance. Same thing about IDE: all current ones have snippet with
auto-switch to the class parents on tab.
All in all, if you are doing exploratory programming, and thus
disposable code, which data classes are fantastic for, inheritance will
keep you in the flow.
2 - it will help sell the data classes
I train a lot of people to Python each year. I never have to explain
classes to people with any kind of programming background. I _always_
have to explain decorators.
People are not used to it, and even kind fear it for quite some time.
Inheritance however, is familiar, and will not only push people to use
data classes more, but also will let them do less mistakes: they know
the danger of parent ordering, but not the ones of decorators ordering.
>
>> - any chance it becomes a built in later ? When classes have been
>> improved in Python 2, the object built-in was added. Imagine if we had
>> had to import it every time... Or maybe just plug it to object like
>> @object.dataclass.
>
> Because of the choice of using module-level functions so as to not
> introduce conflicts in the object's namespace, it would be difficult to
> make this a builtin.
>
> Although now that I think about it, maybe what are currently
> module-level functions should instead be methods on the "dataclass"
> decorator itself:
>
> @dataclass
> class C:
> i: int = dataclass.field(default=1, init=False)
> j: str
>
> c = C('hello')
>
> dataclass.asdict(c)
> {'i': 1, 'j': 'hello'}
>
> Then, "dataclass" would be the only name the module exports, making it
> easier to someday be a builtin. I'm not sure it's important enough for
> this to be a builtin, but this would make it easier. Thoughts? I'm
> usually not a fan of having attributes on a function: it's why
> itertools.chain.from_iterable() is hard to find.
>
> Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mai
