Re: [Python-Dev] PEP 471 -- os.scandir() function -- a better and faster directory iterator

[Eric V. Smith]

On 6/30/2014 10:17 PM, Nick Coghlan wrote:
> 
> On 30 Jun 2014 19:13, "Glenn Linderman"  > wrote:
>>
>>
>> If it is, use ensure_lstat=False, and use the proposed (by me)
> .refresh() API to update the data for those that need it.
> 
> I'm -1 on a refresh API for DirEntry - just use pathlib in that case.

I'm not sure refresh() is the best name, but I think a
"get_stat_info_from_direntry_or_call_stat()" (hah!) makes sense. If you
really need the stat info, then you can write simple code like:

for entry in os.scandir(path):
mtime = entry.get_stat_info_from_direntry_or_call_stat().st_mtime

And it won't call stat() any more times than needed. Once per file on
Posix, zero times per file on Windows.

Without an API like this, you'll need a check in the application code on
whether or not to call stat().

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes & bytearray

[Eric V. Smith]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 1/19/2015 11:36 AM, Ethan Furman wrote:
> I was checking the documentation [1] to see where to put the new
> information about bytes and bytearray %-formatting, and noticed
> that /every/ operation that could modify a bytearray object in
> place (e.g. center, capitalize, strip, etc.) instead returns a new
> copy.
> 
> The section just prior to that [2] does say, "As bytearray objects
> are mutable, they support the mutable sequence operations ...".
> 
> So it seems that when bytearray is treated as ascii data a new
> bytearry is returned, and when bytearray is treated as a container
> it is modified in place:
> 
> New:
> 
> bytearray(b'Chapter Title').center() bytearray(b' Chapter Title
> ').replace(b' ', b'- * - ')
> 
> In-place:
> 
> bytearray(b'abc'][1] = ord(b'z') bytearray(b'def'] += b'ghi' 
> bytearray(b'123'] *= 3
> 
> I now have a minor dilemma:  %-formatting is an ascii operation,
> but it also has an in-place operator (%=) . . . so does %= modify
> the bytearray in place just like += and *= do, or does it return a
> new bytearray just like all the named ascii operations do?  I do
> not know which is more surprising: having one of the in-place
> operators not work in place, or having an unnamed ascii-operation
> not return a new copy.
> 
> Thoughts?

I'd return a new copy (or, not implement it at all and let the default
machinery call __mod__, and do the assignment). That's the least
surprising to me, since the goal is to be compatible with 2.7's str
%-formatting.

- -- 
Eric.
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJUvThwAAoJENxauZFcKtNx/noIAIfIW2Ir4P9bQ9s7bdyJCvMj
RlqfBhWVOjWZ52gK/CdTvpZZDlDUf+gh1JkbvggNvIQHUTy0IY9IxryWvNemfSce
nzZxUhNKzx77f/oGTeGgqBBMZGvCsRCqYFbLBME5LDweiHAIL2MVGqF/SwkobrGq
ruJjSBtNAl28IgpDj7kM7tT7/iXINk6XkVatNa2OxV2FOyYiIz+7Vs2HGpkltzhW
g4qqGEvEpLOl1oRtmI/A3TDFjQgyHc1MKVax6PH/Nq2OMfnoS4hg+jTAzE6Eairh
SmWyZUMjpTeHpCmUgx03WLh8iRTokfE2LN2KJuBN18iAT5EqC6sNTKZm9HX5odw=
=D9zy
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] TypeError messages

[Eric V. Smith]

On 02/20/2015 09:05 AM, Brett Cannon wrote:
> Some messages (only in C) truncate actual type name (%.50s, %.80s,
> %.200s, %.500s). Should type name be truncated at all and for how limit?
> 
> 
> I assume this is over some worry of string size blowing out memory
> allocation or something? If someone can show that's an actual worry then
> fine, otherwise I say don't truncate.

I asked about this years ago, and was told it was in case the type name
pointer was bad, and to limit the amount of garbage printed. Whether
that's an actual problem or not, I can't say. It seems more likely that
you'd get a segfault, but maybe if it was pointing to reused memory it
could be useful.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] how to inspect if something includes a bound first param

[Eric V. Smith]

On 2/24/2015 8:56 PM, Gregory P. Smith wrote:
> inspect.getargspec(method) and inspect.signature(method) both include
> the 'self' parameter but how are we to figure out from method itself
> that it is actually bound and that its first parameter is expected to be
> a bound instance?
> 
> So far I've come up with:
>  /inspect.ismethod(method) or inspect.ismethoddescriptor(method)/
> 
> But that is still not quite right as it remains False in 3.4 for the
> simple case of:
> 
 class A:
> ...  def x(self):
> ...pass
> ...
 inspect.ismethod(A.x)
> False
 inspect.ismethoddescriptor(A.x)
> False
 inspect.signature(A.x).parameters
> mappingproxy(OrderedDict([('self', )]))
 inspect.getargspec(A.x)
> ArgSpec(args=['self'], varargs=None, keywords=None, defaults=None)
> 
> The above works if use it on object.__init__, but not on a method of an
> uninstantiated class.  (it also works on the instantiated A().x)
> 
> Checking if /(inspect.isfunction(method) and method.__qualname__ !=
> method.__name__)/ perhaps but that seems questionably hacky. Is there a
> right way to do this via the inspect module that I've missed?
> 
> It seems like that'd be nice, but I don't feel like I know enough to
> write up a feature request for it yet.  (granted I hope nobody /wants/
> to write code that does this...)

I'm not sure if it's correct, but deep in a library of mine I have:

elif type(fn) == types.MethodType:
# bound method?
if fn.im_self is None:
# no 'self'
nskip = 0
else:
# need to supply 'self'
nskip = 1

This is also a 2.x library. No idea if it works with 3.x.

Eric.





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for comments: [issue22941] IPv4Interface arithmetic changes subnet mask

[Eric V. Smith]

On 3/12/2015 10:00 AM, Antoine Pitrou wrote:
> On Thu, 12 Mar 2015 12:37:16 +
> "Loevborg, Soeren Jakob"  wrote:
>>
> import ipaddress
> ipaddress.IPv4Interface('10.0.0.1/8') + 1
>> IPv4Interface('10.0.0.2/32')
> ipaddress.IPv6Interface('fd00::1/64') + 1
>> IPv6Interface('fd00::2/128')
> 
> Given that the behaviour is unlikely to match any use case, I'd say
> it's a bug.

Agreed.

> Related issue:
> 
 ipaddress.IPv4Interface('10.0.0.255/8') + 1
> IPv4Interface('10.0.1.0/32')
> 
> Should the result be IPv4Interface('10.0.0.0/8')
> (i.e. wrap around)? Should OverflowError be raised?

I think it should be an OverflowError. 10.0.1.0/8 makes no sense, and
10.0.0.0/8 is unlikely to be desirable.

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for comments: [issue22941] IPv4Interface arithmetic changes subnet mask

[Eric V. Smith]

On 3/12/2015 2:17 PM, Antoine Pitrou wrote:
> On Thu, 12 Mar 2015 14:08:01 -0400
> Scott Dial  wrote:
> 
>> On 2015-03-12 10:46 AM, Eric V. Smith wrote:
>>> On 3/12/2015 10:00 AM, Antoine Pitrou wrote:
>>>> Related issue:
>>>>
>>>>>>> ipaddress.IPv4Interface('10.0.0.255/8') + 1
>>>> IPv4Interface('10.0.1.0/32')
>>>>
>>>> Should the result be IPv4Interface('10.0.0.0/8')
>>>> (i.e. wrap around)? Should OverflowError be raised?
>>>
>>> I think it should be an OverflowError. 10.0.1.0/8 makes no sense, and
>>> 10.0.0.0/8 is unlikely to be desirable.
>>>
>>
>> No, you are both imbuing meaning to the dot-notation that is not there.
>> A.B.C.D is just an ugly way to represent a single 32-bit unsigned
>> integer. The only contentious part would be when your addition would
>> overlfow the host-portion of the address:
>>
>>>>> ipaddress.IPv4Interface('10.255.255.255/8') + 1
>> IPv4Interface('11.0.0.0/8')
> 
> Oh, well, you're right, I borked my subnet. So, for the record, I meant
> "10.0.0.255/24" where the issue actually appears.

I, too, was thinking /24. I think that overflowing the host portion
should raise OverflowError.

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for comments: [issue22941] IPv4Interface arithmetic changes subnet mask

[Eric V. Smith]

On 3/14/2015 7:04 AM, francis wrote:
> Hi,
>> I, too, was thinking /24. I think that overflowing the host portion
>> should raise OverflowError.
>>
> Just curiosity, why not a modulo calculation on the subnet instead
> of raising the error?

Personally, I can't imaging wanting that behavior. I can't say I've ever
needed additional at all with an IP address, but if I did, it would only
be to stay within the host portion. To wrap around seems odd. If you had
a /24 address and added 1000, should that really be the same as adding
232 (= 1000 % 256)? It seems more likely it's an error.

I'm +0 on adding addition with an OverflowError for the host portion,
and -1 on addition with modulo arithmetic.

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Request for comments: [issue22941] IPv4Interface arithmetic changes subnet mask

[Eric V. Smith]

On 3/14/2015 4:52 PM, Eric V. Smith wrote:
> On 3/14/2015 7:04 AM, francis wrote:
>> Hi,
>>> I, too, was thinking /24. I think that overflowing the host portion
>>> should raise OverflowError.
>>>
>> Just curiosity, why not a modulo calculation on the subnet instead
>> of raising the error?
> 
> Personally, I can't imaging wanting that behavior. I can't say I've ever
> needed additional at all with an IP address, but if I did, it would only

That should be "I can't say I've ever needed addition at all...".

> be to stay within the host portion. To wrap around seems odd. If you had
> a /24 address and added 1000, should that really be the same as adding
> 232 (= 1000 % 256)? It seems more likely it's an error.
> 
> I'm +0 on adding addition with an OverflowError for the host portion,
> and -1 on addition with modulo arithmetic.

-- 
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Keyword-only parameters

[Eric V. Smith]

I'm working on adding a numeric_owner parameter to some tarfile methods
(http://bugs.python.org/issue23193),

In a review, Berker suggested making the parameter keyword-only. I agree
that you'd likely never want to pass just "True", but that
"numeric_owner=True" would be a better usage.

But, I don't see a lot of keyword-only parameters being added to stdlib
code. Is there some position we've taken on this? Barring someone saying
"stdlib APIs shouldn't contain keyword-only params", I'm inclined to
make numeric_owner keyword-only.

Is there anything stopping me from making it keyword-only?

Thanks.
Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Keyword-only parameters

[Eric V. Smith]

On 04/14/2015 02:11 PM, Zachary Ware wrote:
> On Tue, Apr 14, 2015 at 1:06 PM, Steven D'Aprano  wrote:
>> On Tue, Apr 14, 2015 at 01:40:40PM -0400, Eric V. Smith wrote:
>>> But, I don't see a lot of keyword-only parameters being added to stdlib
>>> code. Is there some position we've taken on this? Barring someone saying
>>> "stdlib APIs shouldn't contain keyword-only params", I'm inclined to
>>> make numeric_owner keyword-only.
>>
>> I expect that's because keyword-only parameters are quite recent (3.x
>> only) and most of the stdlib is quite old.
>>
>> Keyword-only feels right for this to me too.
> 
> +1

Thanks. I'll make it keyword-only.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Summary of Python tracker Issues

[Eric V. Smith]

Thank you, David, for all of your work. It's much appreciated. 

--
Eric.

> On Apr 17, 2015, at 7:18 PM, R. David Murray  wrote:
> 
>> On Fri, 17 Apr 2015 18:08:24 +0200, Python tracker  
>> wrote:
>> 
>> ACTIVITY SUMMARY (2015-04-10 - 2015-04-17)
>> Python tracker at http://bugs.python.org/
>> 
>> To view or respond to any of the issues listed below, click on the issue.
>> Do NOT respond to this message.
>> 
>> Issues counts and deltas:
>>  open4792 (-31)
>>  closed 30957 (+113)
>>  total  35749 (+82)
> 
> That's a successful sprint week :)
> 
> Thanks everyone!
> 
> --David
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
> 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-checkins] cpython: Issue #24064: Property() docstrings are now writeable.

[Eric V. Smith]

On 5/13/2015 4:10 AM, raymond.hettinger wrote:
> https://hg.python.org/cpython/rev/1e8a768fa0a5
> changeset:   96010:1e8a768fa0a5
> user:Raymond Hettinger 
> date:Wed May 13 01:09:59 2015 -0700
> summary:
>   Issue #24064: Property() docstrings are now writeable.
> (Patch by Berker Peksag.)

> diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst
> --- a/Doc/library/collections.rst
> +++ b/Doc/library/collections.rst
> @@ -924,6 +924,15 @@
>  
>  >>> Point3D = namedtuple('Point3D', Point._fields + ('z',))
>  
> +Docstrings can be customized by making direct assignments to the ``__doc__``
> +fields:
> +
> +   >>> Book = namedtuple('Book', ['id', 'title', 'authors'])
> +   >>> Book.__doc__ = 'Hardcover book in active collection'
> +   >>> Book.id = '13-digit ISBN'
> +   >>> Book.title = 'Title of first printing'
> +   >>> Book.author = 'List of authors sorted by last name'

Should these be:
Book.id.__doc__ = ...
etc.?

> +Point = namedtuple('Point', ['x', 'y'])
> +Point.__doc__ = 'ordered pair'
> +Point.x.__doc__ = 'abscissa'
> +Point.y.__doc__ = 'ordinate'

These lines from /Doc/whatsnew/3.5.rst would make me think so.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

I've written a PEP for what might be thought of as "mutable namedtuples 
with defaults, but not inheriting tuple's behavior" (a mouthful, but it 
sounded simpler when I first thought of it). It's heavily influenced by 
the attrs project. It uses PEP 526 type annotations to define fields. 
From the overview section:


@dataclass
class InventoryItem:
name: str
unit_price: float
quantity_on_hand: int = 0

def total_cost(self) -> float:
return self.unit_price * self.quantity_on_hand

Will automatically add these methods:

  def __init__(self, name: str, unit_price: float, quantity_on_hand: 
int = 0) -> None:

  self.name = name
  self.unit_price = unit_price
  self.quantity_on_hand = quantity_on_hand
  def __repr__(self):
  return 
f'InventoryItem(name={self.name!r},unit_price={self.unit_price!r},quantity_on_hand={self.quantity_on_hand!r})'

  def __eq__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) == 
(other.name, other.unit_price, other.quantity_on_hand)

  return NotImplemented
  def __ne__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) != 
(other.name, other.unit_price, other.quantity_on_hand)

  return NotImplemented
  def __lt__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) < 
(other.name, other.unit_price, other.quantity_on_hand)

  return NotImplemented
  def __le__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) <= 
(other.name, other.unit_price, other.quantity_on_hand)

  return NotImplemented
  def __gt__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) > 
(other.name, other.unit_price, other.quantity_on_hand)

  return NotImplemented
  def __ge__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) >= 
(other.name, other.unit_price, other.quantity_on_hand)

  return NotImplemented

Data Classes saves you from writing and maintaining these functions.

The PEP is largely complete, but could use some filling out in places. 
Comments welcome!


Eric.

P.S. I wrote this PEP when I was in my happy place.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

Oops, I forgot the link. It should show up shortly at 
https://www.python.org/dev/peps/pep-0557/.


Eric.

On 9/8/17 7:57 AM, Eric V. Smith wrote:

I've written a PEP for what might be thought of as "mutable namedtuples
with defaults, but not inheriting tuple's behavior" (a mouthful, but it
sounded simpler when I first thought of it). It's heavily influenced by
the attrs project. It uses PEP 526 type annotations to define fields.
From the overview section:

@dataclass
class InventoryItem:
name: str
unit_price: float
quantity_on_hand: int = 0

def total_cost(self) -> float:
return self.unit_price * self.quantity_on_hand

Will automatically add these methods:

  def __init__(self, name: str, unit_price: float, quantity_on_hand: int
= 0) -> None:
  self.name = name
  self.unit_price = unit_price
  self.quantity_on_hand = quantity_on_hand
  def __repr__(self):
  return
f'InventoryItem(name={self.name!r},unit_price={self.unit_price!r},quantity_on_hand={self.quantity_on_hand!r})'

  def __eq__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) ==
(other.name, other.unit_price, other.quantity_on_hand)
  return NotImplemented
  def __ne__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) !=
(other.name, other.unit_price, other.quantity_on_hand)
  return NotImplemented
  def __lt__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) <
(other.name, other.unit_price, other.quantity_on_hand)
  return NotImplemented
  def __le__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) <=
(other.name, other.unit_price, other.quantity_on_hand)
  return NotImplemented
  def __gt__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) >
(other.name, other.unit_price, other.quantity_on_hand)
  return NotImplemented
  def __ge__(self, other):
  if other.__class__ is self.__class__:
  return (self.name, self.unit_price, self.quantity_on_hand) >=
(other.name, other.unit_price, other.quantity_on_hand)
  return NotImplemented

Data Classes saves you from writing and maintaining these functions.

The PEP is largely complete, but could use some filling out in places.
Comments welcome!

Eric.

P.S. I wrote this PEP when I was in my happy place.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

Hah! Thanks for the catch. 

There is no C in my happy place. 

--
Eric.

> On Sep 8, 2017, at 8:52 AM, Paul Moore  wrote:
> 
>> On 8 September 2017 at 15:57, Eric V. Smith  wrote:
>> I've written a PEP for what might be thought of as "mutable namedtuples with
>> defaults, but not inheriting tuple's behavior" (a mouthful, but it sounded
>> simpler when I first thought of it). It's heavily influenced by the attrs
>> project. It uses PEP 526 type annotations to define fields.
> [...]
>> The PEP is largely complete, but could use some filling out in places.
>> Comments welcome!
>> 
>> Eric.
>> 
>> P.S. I wrote this PEP when I was in my happy place.
> 
> Looks good! One minor point - apparently in your happy place, C and
> Python have the same syntax :-)
> 
> """
> field's may optionally specify a default value, using normal Python syntax:
> 
> @dataclass
> class C:
>int a   # 'a' has no default value
>int b = 0   # assign a default value for 'b'
> """

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/8/17 9:07 AM, Eric V. Smith wrote:

Hah! Thanks for the catch.


I've pushed a fix to the PEP repo, it should show up online shortly.

Eric.



There is no C in my happy place.

--
Eric.


On Sep 8, 2017, at 8:52 AM, Paul Moore  wrote:


On 8 September 2017 at 15:57, Eric V. Smith  wrote:
I've written a PEP for what might be thought of as "mutable namedtuples with
defaults, but not inheriting tuple's behavior" (a mouthful, but it sounded
simpler when I first thought of it). It's heavily influenced by the attrs
project. It uses PEP 526 type annotations to define fields.

[...]

The PEP is largely complete, but could use some filling out in places.
Comments welcome!

Eric.

P.S. I wrote this PEP when I was in my happy place.


Looks good! One minor point - apparently in your happy place, C and
Python have the same syntax :-)

"""
field's may optionally specify a default value, using normal Python syntax:

@dataclass
class C:
   int a   # 'a' has no default value
   int b = 0   # assign a default value for 'b'
"""


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/8/17 3:20 PM, Mike Miller wrote:


On 2017-09-08 07:57, Eric V. Smith wrote:

I've written a PEP for…


Apologies for the following list dumb questions and bikesheds:


- 'Classes can be thought of as "mutable namedtuples with defaults".'

  - A C/C++ (struct)ure sounds like a simpler description that many more
would understand.


Yes, other people have pointed out that this might not be the best 
"elevator pitch" example. I'm thinking about it.



- dataclass name:

- class, redundant
- data, good but very common
- struct, used?
- Record?  (best I could come up with)


There was a bunch of discussions on this. We're delaying the name 
bikeshedding for later (and maybe never).



- Source needs blanks between functions, hard to read.


It's supposed to be hard to read! You're just supposed to think "am I 
glad I don't have to read or write that". But I'll look at it.



- Are types required?


Annotations are required, the typing module is not.


  Maybe an example or two with Any?


I'd rather leave it like it is: typing is referenced only once, for 
ClassVar.



- Intro discounts inheritance and metaclasses as "potentially interfering",
  but unclear why that would be the case.
  Inheritance is easy to override, metaclasses not sure?


I don't really want to get in to the history of why people don't like 
inheritance, single and multi. Or how metaclass magic can make life 
difficult. I just want to point out that Data Classes don't interfere at 
all.



- Perhaps mention ORMs/metaclass approach, as prior art:

  https://docs.djangoproject.com/en/dev/topics/db/models/

- Perhaps mention Kivy Properties, as prior art:

  https://kivy.org/docs/api-kivy.properties.html


Those are all good. Thanks.


- For mutable default values:

  @dataclass
  class C:
  x: list  # = field(default_factory=list)

  Could it detect list as a mutable class "type" and set it as a factory
  automatically?

  The PEP/bug #3 mentions using copy, but that's not exactly what I'm
asking
  above.


The problem is: how do you know what's a mutable type? There's no 
general way to know. The behavior in the PEP is just mean to stop the 
worst of it.


I guess we could have an option that says: call the type to create a 
new, empty instance.


@dataclass
class C:
x: list = field(default_type_is_factory=True)


Thanks for the critical reading and your comments. I'm going to push a 
new version early next week, when I get back from traveling.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/9/2017 11:41 AM, Gustavo Carneiro wrote:

Hi, it is not clear whether anything is done to total_cost:

     def total_cost(self) -> float:

Does this become a property automatically, or is it still a method 
call?  To that end, some examples of *using* a data class, not just 
defining one, would be helpful.


If it remains a normal method, why put it in this example at all? Makes 


Nothing is done with total_cost, it's still a method. It's meant to show 
that you can use methods in a Data Class.  Maybe I should add a method 
that has a parameter, or at least explain why that method is present in 
the example.


I'm not sure how I'd write an example showing you can you do everything 
you can with an undecorated class. I think some text explanation would 
be better.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/10/2017 10:00 AM, Michel Desmoulin wrote:

The reaction is overwhelmingly positive everywhere: hacker news, reddit,
twitter.


Do you have a pointer to the Hacker News discussion? I missed it.


People have been expecting something like that for a long time.


Me, too!


3 questions:

- is providing validation/conversion hooks completely out of the
question of still open for debate ? I know it's to keep the
implementation simple but have a few callbacks run in the __init__ in a
foo loop is not that much complexity. You don't have to provide
validators, but having a validators parameters on field() would be a
huge time saver. Just a list of callables called when the value is first
set, potentially raising an exception that you don't even need to
process in any way. It returns the value converted, and voilà. We all do
that every day manually.


I don't particularly want to add validation specifically. I want to make 
it possible to add validation yourself, or via a library.


What I think I'll do is add a metadata parameter to fields(), defaulting 
to None. Then you could write a post-init hook that does whatever 
single- and multi-field validations you want (or whatever else you want 
to do). Although this plays poorly with "frozen" classes: it's always 
something! I'll think about it.


To make this most useful, I need to get the post-init hook to take an 
optional parameter so you can get data to it. I don't have a good way to 
do this, yet. Suggestions welcomed.


Although if the post-init hook takes a param that you can pass in at 
object creation time, I guess there's really no need for a per-field 
metadata parameter: you could use the field name as a key to look up 
whatever you wanted to know about the field.



- I read Guido talking about some base class as alternative to the
generator version, but don't see it in the PEP. Is it still considered ?


I'm going to put some words in explaining why I don't want to use base 
classes (I don't think it buys you anything). Do you have a reason for 
preferring base classes?



- any chance it becomes a built in later ? When classes have been
improved in Python 2, the object built-in was added. Imagine if we had
had to import it every time... Or maybe just plug it to object like
@object.dataclass.


Because of the choice of using module-level functions so as to not 
introduce conflicts in the object's namespace, it would be difficult to 
make this a builtin.


Although now that I think about it, maybe what are currently 
module-level functions should instead be methods on the "dataclass" 
decorator itself:


@dataclass
class C:
  i: int = dataclass.field(default=1, init=False)
  j: str

c = C('hello')

dataclass.asdict(c)
{'i': 1, 'j': 'hello'}

Then, "dataclass" would be the only name the module exports, making it 
easier to someday be a builtin. I'm not sure it's important enough for 
this to be a builtin, but this would make it easier. Thoughts? I'm 
usually not a fan of having attributes on a function: it's why 
itertools.chain.from_iterable() is hard to find.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/8/2017 11:01 AM, Eric V. Smith wrote:
Oops, I forgot the link. It should show up shortly at 
https://www.python.org/dev/peps/pep-0557/.


And now I've pushed a version that works with Python 3.6 to PyPI at 
https://pypi.python.org/pypi/dataclasses


It implements the PEP as it currently stands. I'll be making some tweaks 
in the coming weeks. Feedback is welcomed.


The repo is at https://github.com/ericvsmith/dataclasses

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

> On Sep 10, 2017, at 5:13 PM, Mike Miller  wrote:
> 
> Thanks for the info.
> 
> On 2017-09-08 15:40, Eric V. Smith wrote:
>>> - Are types required?
>> Annotations are required, the typing module is not.
>>>  Maybe an example or two with Any?
>> I'd rather leave it like it is: typing is referenced only once, for ClassVar.
> 
> 
> Guess I really meant "object" or "type" above, not "typing.Any."
> 
> For a decade or two, my use of structs/records in Python has been 
> dynamically-typed and that hasn't been an issue.  As the problem this PEP is 
> solving is orthogonal to typing improvements, it feels a bit like typing is 
> being coupled and pushed with it, whether wanted or not.
> 
> Not that I dislike typing mind you (appreciated on projects with large 
> teams), it's just as mentioned not related to the problem of class definition 
> verbosity or lacking functionality.

Using type annotations buys two things. 

One is the concise syntax: it's how the decorator finds the fields. In the 
simple case, it's what let's you define a data class without using the call to 
fields(), which is analogous to attrs's attr.ib(). 

The other thing it buys you is compatibility with type checkers. As Ivan said, 
the design was very careful to be compatible with type checkers. 

So other than requiring some type as an annotation, there's no dependency added 
for typing in the genera sense. You can use a type of object if you want, or 
just lie to it and say None (though I'm not recommending that!). It's 
completely ignored at runtime (except for the ClassVar case, as the PEP states).

I'll think about adding some more language to the PEP about it.

Eric. 
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/10/17 10:27 PM, Raymond Hettinger wrote:



On Sep 10, 2017, at 4:54 PM, Eric V. Smith  wrote:

And now I've pushed a version that works with Python 3.6 to PyPI at 
https://pypi.python.org/pypi/dataclasses

It implements the PEP as it currently stands. I'll be making some tweaks in the 
coming weeks. Feedback is welcomed.

The repo is at https://github.com/ericvsmith/dataclasses


+1
Overall, this looks very well thought out.
Nice work!


Thank you.


Once you get agreement on the functionality, name bike-shedding will likely be 
next.  In a way, all classes are data classes so that name doesn't tell me 
much.  Instead, it would be nice to have something suggestive of what it 
actually does which is automatically adding boilerplate methods to a general 
purpose class.  Perhaps, @boilerplate or @autoinit or some such.


There was some discussion on naming at 
https://github.com/ericvsmith/dataclasses/issues/12.


In that issue, Guido said use Data Classes (the concept), dataclasses 
(the module) and dataclass (the decorator). I think if someone came up 
with an awesomely better name, we could all be convinced to use it. But 
so far, nothing's better.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/10/17 11:08 PM, Nathaniel Smith wrote:

Hi Eric,

A few quick comments:

Why do you even have a hash= argument on individual fields? For the
whole class, I can imagine you might want to explicitly mark a whole
class as unhashable, but it seems like the only thing you can do with
the field-level hash= argument is to create a class where the __hash__
and __eq__ take different fields into account, and why would you ever
want that?


The use case is that you have a cache, or something similar, that 
doesn't affect the object identity.



Though honestly I can see a reasonable argument for removing the
class-level hash= option too. And even if you keep it you might want to
error on some truly nonsensical options like defining __hash__ without
__eq__. (Also watch out that Python's usual rule about defining __eq__
blocking the inheritance of __hash__ does not kick in if __eq__ is added
after the class is created.)

I've sometimes wished that attrs let me control whether it generated
equality methods (eq/ne/hash) separately from ordering methods
(lt/gt/...). Maybe the cmp= argument should take an enum with options
none/equality-only/full?


Yeah, I've thought about this, too. But I don't have any use case in 
mind, and if it hasn't come up with attrs, then I'm reluctant to break 
new ground here.



The "why not attrs" section kind of reads like "because it's too popular
and useful"?


I'll add some words to that section, probably focused on typing 
compatibility. My general feeling is that attrs has some great design 
decisions, but goes a little too far (e.g., conversions, validations). 
As with most things we add, I'm trying to be as minimalist as possible, 
while still being widely useful and allowing 3rd party extensions and 
future features.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/11/17 9:43 AM, tds...@mailbox.org wrote:

Hi Eric,

I have on question not addressed yet.
The implementation is based on "__annotations__" where the type is specified.
But "__annotations__" is not always filled. An interpreter version with special 
optimization
could remove all __annotations__ for performance reasons. (Discussed in other 
threads)

In this case the dataclass does not work or will there be a fallback?

I know it is a little bit hypothetical because an interpreter with this 
optimization is
not there yet. I am looking only in the future a bit.
Asking this because type annotations are stated as completely optional for 
Python.
And this use case will break this assumption.


Yes, if there are no __annotations__, then Data Classes would break. 
typing.NamedTuple has the same issue. We discussed it a little bit last 
week, but I don't think we came to any conclusions.


Since @dataclass ignores the value of the annotation (except for 
typing.ClassVar), it would continue to work if the type was present, buy 
maybe mapped to None or similar.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/11/2017 12:34 PM, Nathaniel Smith wrote:

On Mon, Sep 11, 2017 at 5:32 AM, Eric V. Smith  wrote:

On 9/10/17 11:08 PM, Nathaniel Smith wrote:

Why do you even have a hash= argument on individual fields? For the
whole class, I can imagine you might want to explicitly mark a whole
class as unhashable, but it seems like the only thing you can do with
the field-level hash= argument is to create a class where the __hash__
and __eq__ take different fields into account, and why would you ever
want that?



The use case is that you have a cache, or something similar, that doesn't
affect the object identity.


But wouldn't this just be field(cmp=False), no need to fiddle with hash=?


Ah, true. You're right, I can't see any good use for setting hash on a 
field that isn't already controlled by cmp. I think field level hash can go.



Though honestly I can see a reasonable argument for removing the
class-level hash= option too. And even if you keep it you might want to
error on some truly nonsensical options like defining __hash__ without
__eq__. (Also watch out that Python's usual rule about defining __eq__
blocking the inheritance of __hash__ does not kick in if __eq__ is added
after the class is created.)


At the class level, I think it makes more sense. But I'll write up some 
motivating examples.

I've sometimes wished that attrs let me control whether it generated
equality methods (eq/ne/hash) separately from ordering methods
(lt/gt/...). Maybe the cmp= argument should take an enum with options
none/equality-only/full?



Yeah, I've thought about this, too. But I don't have any use case in mind,
and if it hasn't come up with attrs, then I'm reluctant to break new ground
here.


https://github.com/python-attrs/attrs/issues/170

 From that thread, it feels like part of the problem is that it's
awkward to encode this using only boolean arguments, but they're sort
of stuck with that for backcompat with how they originally defined
cmp=, hence my suggestion to consider making it an enum from the start
:-).


I'll respond to other emails about this, probably tomorrow.

Eric.


The "why not attrs" section kind of reads like "because it's too popular
and useful"?



I'll add some words to that section, probably focused on typing
compatibility. My general feeling is that attrs has some great design
decisions, but goes a little too far (e.g., conversions, validations). As
with most things we add, I'm trying to be as minimalist as possible, while
still being widely useful and allowing 3rd party extensions and future
features.


If the question is "given that we're going to add something to the
stdlib, why shouldn't that thing be attrs?" then I guess it's
sufficient to say "because the attrs developers didn't want it". But I
think the PEP should also address the question "why are we adding
something to the stdlib, instead of just recommending people install
attrs".

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/11/2017 6:28 PM, Guido van Rossum wrote:
Oddly I don't like the enum (flag names get too long that way), but I do 
agree with everything else Barry said (it should be a trivalue flag and 
please don't name it cmp).


So if we don't do enums, I think the choices are ints, strs, or maybe 
True/False/None. Do you have a preference here?


If int or str, I assume we'd want module-level constants.

I like the name compare=, and 3 values makes sense: None, Equality, Ordered.

Eric.



On Mon, Sep 11, 2017 at 3:16 PM, Ethan Furman > wrote:


On 09/11/2017 03:00 PM, Barry Warsaw wrote:

On Sep 10, 2017, at 20:08, Nathaniel Smith wrote:


I've sometimes wished that attrs let me control whether it
generated equality methods (eq/ne/hash) separately from
ordering methods (lt/gt/...). Maybe the cmp= argument should
take an enum with options none/equality-only/full?


I have had use cases where I needed equality comparisons but not
ordered comparisons, so I’m in favor of the option to split
them.  (atm, I can’t bring up a specific case, but it’s not
uncommon.)

Given that you only want to support the three states that
Nathaniel describes, I think an enum makes the most sense, and
it certainly would read well.  I.e. there’s no sense in
supporting the ordered comparisons and not equality, so that’s
not a state that needs to be represented.

I’d make one other suggestion here: please let’s not call the
keyword `cmp`.  That’s reminiscent of Python 2’s `cmp` built-in,
which of course doesn’t exist in Python 3.  Using `cmp` is just
an unnecessarily obfuscating abbreviation.  I’d suggest just
`compare` with an enum like so:

enum Compare(enum.Enum):
      none = 1
      unordered = 2
      ordered = 3


I like the enum idea (suprise! ;) but I would suggest "equal" or
"equivalent" instead of "unordered"; better to say what they are
rather than what they are not.

--
~Ethan~

___
Python-Dev mailing list
Python-Dev@python.org 
https://mail.python.org/mailman/listinfo/python-dev

Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org 





--
--Guido van Rossum (python.org/~guido )


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 9/10/2017 11:08 PM, Nathaniel Smith wrote:

Hi Eric,

A few quick comments:

Why do you even have a hash= argument on individual fields? For the 
whole class, I can imagine you might want to explicitly mark a whole 
class as unhashable, but it seems like the only thing you can do with 
the field-level hash= argument is to create a class where the __hash__ 
and __eq__ take different fields into account, and why would you ever 
want that?


See the discussion at 
https://github.com/ericvsmith/dataclasses/issues/44 for why we're 
keeping the field-level hash function.


Quick version, from Guido: "There's a legitimate reason for having a 
field in the eq but not in the hash. After all hash is always followed 
by an eq check and it is totally legit for the eq check to say two 
objects are not equal even though their hashes are equal."


This would be in the case where a field should be used for equality 
testing, but computing its hash is expensive.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Investigating time for `import requests`

[Eric V. Smith]

The easiest workaround at the moment is still pretty clumsy:

def import_SLLError():
     from requests.exceptions import SLLError
     return SLLError

...


     except import_SLLError():


But what happens if that gives you an ImportError?


You can't catch a requests exception unless requests has already been 
imported, you could do something like:


except Exception as ex:
if 'requests' in sys.modules:
import requests  # this is basically free at this point
if isinstance(ex, requests.exceptions):
...

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557: Data Classes

[Eric V. Smith]

On 10/12/2017 6:33 PM, Chris Barker wrote:
On Thu, Oct 12, 2017 at 3:20 AM, Steve Holden > wrote:


  The reason I liked "row" as a name is because it resembles 
"vector" and hence is loosely assocaited with the concept of a tuple

as well as being familiar to database users. In fact the answer to a
relational query was, I believe, originally formally defined as a
set of tuples.


Is the intent that these things preserve order?


In the sense that the parameters to __init__(), the appearance in the 
repr, the order of the returned tuple in as_tuple(), and the order of 
comparisons will be the same as the order that the fields are defined, 
then yes.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] The syntax of replacement fields in format strings

[Eric V. Smith]

If I had it to do over again, I’d implement it more strictly and only allow 
chars that are valid in identifiers. 

But see https://bugs.python.org/issue31907 for a case that is currently valid 
and would break if we changed how it worked. 

I’m not sure it’s worth the churn of deprecating this and eventually making it 
illegal. 

--
Eric.

> On Oct 31, 2017, at 6:37 AM, Serhiy Storchaka  wrote:
> 
> According to the specification of format string syntax [1] (I meant 
> str.format(), not f-strings), both argument name and attribute name must be 
> Python identifiers.
> 
> But the current implementation is more lenient and allow arbitrary sequences 
> of characters while they don't contain '.', '[', ']', '{', '}', ':', '!'.
> 
> >>> '{#}'.format_map({'#': 42})
> '42'
> >>> import types
> >>> '{0.#}'.format(types.SimpleNamespace(**{'#': 42}))
> '42'
> 
> This can be confusing due to similarity with the format string syntaxes in 
> str.format() and f-strings.
> 
> >> name = 'abc'
> >>> f'{name.upper()}'
> 'ABC'
> >>> '{name.upper()}'.format(name='abc')
> Traceback (most recent call last):
>  File "", line 1, in 
> AttributeError: 'str' object has no attribute 'upper()'
> 
> If accept only identifiers, we could produce more specific error message.
> 
> Is there a bug in the documentation or in the implementation?
> 
> [1] https://docs.python.org/3/library/string.html#format-string-syntax
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Reminder: 12 weeks to 3.7 feature code cutoff

[Eric V. Smith]

On 11/1/2017 5:47 PM, Ned Deily wrote:

Happy belated Halloween to those who celebrate it; I hope it wasn't too scary!  
Also possibly scary: we have just a little over 12 weeks remaining until Python 
3.7's feature code cutoff, 2018-01-29.  Those 12 weeks include a number of 
traditional holidays around the world so, if you are planning on writing 
another PEP for 3.7 or working on getting an existing one approved or getting 
feature code reviewed, please plan accordingly.If you have something in 
the pipeline, please either let me know or, when implemented, add the feature 
to PEP 537, the 3.7 Release Schedule PEP.


I hope to be able to free up some time to complete PEP 557, Data Classes.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Remove typing from the stdlib

[Eric V. Smith]

On 11/3/2017 12:15 PM, Victor Stinner wrote:

Hi,

2017-11-03 15:36 GMT+01:00 Guido van Rossum :

Maybe we should remove typing from the stdlib?
https://github.com/python/typing/issues/495



The typing module is not used yet in the stdlib, so there is no
technically reason to keep typing part of the stdlib. IMHO it's
perfectly fine to keep typing and annotations out of the stdlib, since
the venv & pip tooling is now rock solid ;-)


I'm planning on using it for PEP 557:
https://www.python.org/dev/peps/pep-0557/#class-variables

The way the code currently checks for this should still work if typing 
is not in the stdlib, although of course it's assuming that the name 
"typing" really is the "official" typing library.


# If typing has not been imported, then it's impossible for
#  any annotation to be a ClassVar. So, only look for ClassVar
#  if typing has been imported.
typing = sys.modules.get('typing')
if typing is not None:
# This test uses a typing internal class, but it's the best
#  way to test if this is a ClassVar.
if type(a_type) is typing._ClassVar:
# This field is a ClassVar. Ignore it.
continue

See also https://github.com/ericvsmith/dataclasses/issues/14

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Allow annotations using basic types in the stdlib?

[Eric V. Smith]

On 11/6/2017 1:40 PM, Barry Warsaw wrote:

On Nov 6, 2017, at 08:02, Victor Stinner  wrote:


While discussions on the typing module are still hot, what do you
think of allowing annotations in the standard libraries, but limited
to a few basic types:


I’m still -1 on adding annotations to the stdlib, despite their increasing use 
out in the wild, for the reasons that Steve and David have pointed out.  (Let’s 
let Eric be the one that breaks the mold with data classes.  Then we can blame 
him!)


Thanks for volunteering me!

Note that dataclasses completely ignores the annotations (they could all 
be None, for all I care), except for the one specific case of ClassVar. 
And that's still up for discussion, see 
https://github.com/ericvsmith/dataclasses/issues/61.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: go back to enabling DeprecationWarning by default

[Eric V. Smith]

On 11/6/2017 1:12 PM, Barry Warsaw wrote:

On Nov 5, 2017, at 20:47, Nick Coghlan  wrote:


warnings.silence_deprecations()
python -X silence-deprecations
PYTHONSILENCEDEPRECATIONS=x


It could be interesting to combine this with Tim's suggestion of
putting an upper version limit on the silencing, so the above may look
like:

warnings.ignore_deprecations((3, 7))
python -X ignore-deprecations=3.7
PYTHONIGNOREDEPRECATIONS=3.7


That could be cool as long as we also support wildcards, e.g. defaults along 
the lines of my suggestions above to ignore everything.


I'd like to see a command line or environment variable that says: "turn 
on deprecation warnings (and/or pending deprecation warnings), but do 
not show warnings for this list of modules (possibly regex's)".


Like:
PYTHONDEPRECATIONWARNINGSEXCEPTFOR=PIL,requests.*

Then I'd just turn it on for all modules (empty string?), and when I got 
something that was flooding me with output I'd add it to the list.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

The updated version should show up at 
https://www.python.org/dev/peps/pep-0557/ shortly.


The major changes from the previous version are:

- Add InitVar to specify initialize-only fields.
- Renamed __dataclass_post_init__() to __post_init().
- Rename cmp to compare.
- Added eq, separate from compare, so you can test
  unorderable items for equality.
- Flushed out asdict() and astuple().
- Changed replace() to just call __init__(), and dropped
  the complex post-create logic.

The only open issues I know of are:
- Should object comparison require an exact match on the type?
  https://github.com/ericvsmith/dataclasses/issues/51
- Should the replace() function be renamed to something else?
  https://github.com/ericvsmith/dataclasses/issues/77

Most of the items that were previously discussed on python-dev were 
discussed in detail at https://github.com/ericvsmith/dataclasses. Before 
rehashing an old discussion, please check there first.


Also at https://github.com/ericvsmith/dataclasses is an implementation, 
with tests, that should work with 3.6 and 3.7. The only action item for 
the code is to clean up the implementation of InitVar, but that's 
waiting for PEP 560. Oh, and if PEP 563 is accepted I'll also need to do 
some work.


Feedback is welcomed!

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

One more change:

 - Per-field metadata, for use by third parties.

Also, thanks to Guido and Ivan for all of their feedback on the various 
issues that got the PEP to this point.


Eric.

On 11/25/2017 4:06 PM, Eric V. Smith wrote:
The updated version should show up at 
https://www.python.org/dev/peps/pep-0557/ shortly.


The major changes from the previous version are:

- Add InitVar to specify initialize-only fields.
- Renamed __dataclass_post_init__() to __post_init().
- Rename cmp to compare.
- Added eq, separate from compare, so you can test
   unorderable items for equality.
- Flushed out asdict() and astuple().
- Changed replace() to just call __init__(), and dropped
   the complex post-create logic.

The only open issues I know of are:
- Should object comparison require an exact match on the type?
   https://github.com/ericvsmith/dataclasses/issues/51
- Should the replace() function be renamed to something else?
   https://github.com/ericvsmith/dataclasses/issues/77

Most of the items that were previously discussed on python-dev were 
discussed in detail at https://github.com/ericvsmith/dataclasses. Before 
rehashing an old discussion, please check there first.


Also at https://github.com/ericvsmith/dataclasses is an implementation, 
with tests, that should work with 3.6 and 3.7. The only action item for 
the code is to clean up the implementation of InitVar, but that's 
waiting for PEP 560. Oh, and if PEP 563 is accepted I'll also need to do 
some work.


Feedback is welcomed!

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/26/2017 12:23 AM, Carl Meyer wrote:

Hi Eric,

Really excited about this PEP, thanks for working on it. 


Thanks, Carl.


A couple minor
questions:


If compare is True, then eq is ignored, and __eq__ and __ne__ will be

automatically generated.

IMO it's generally preferable to make nonsensical parameter combinations
an immediate error, rather than silently ignore one of them. Is there a
strong reason for letting nonsense pass silently here?

(I reviewed the previous thread; there was a lot of discussion about
enums/flags vs two boolean params, but I didn't see explicit discussion
of this issue; the only passing references I noticed said the invalid
combo should be "disallowed", e.g. Guido in [1], which to me implies "an
error.")


I think you're right here. I'll change it to a ValueError.


isdataclass(instance): Returns True if instance is an instance of a

Data Class, otherwise returns False.

Something smells wrong with the naming here. If I have

 @dataclass
 class Person:
 name: str

I think it would be considered obvious and undeniable (in English prose,
anyway) that Person is a dataclass. So it seems wrong to have
`isdataclass(Person)` return `False`. Is there a reason not to let it
handle either a class or an instance (looks like it would actually
simplify the implementation)?


I think of this as really "isdataclassinstance". Let's see what others 
think. There are places in dataclasses.py where I need to know both 
ways: for example fields() works with either a class or instance, but 
asdict() just an instance.


For what it's worth, the equivalent attrs API, attr.has(), returns True 
for both an instance and a class. And the recommended solution for a 
namedtuple (check for existence of _fields) would also work for an 
instance and a class. And I suppose it's easy enough for the caller to 
further disallow a class, if that's what they want.


Eric.



Carl


  [1] https://mail.python.org/pipermail/python-dev/2017-September/149505.html

On 11/25/2017 01:06 PM, Eric V. Smith wrote:

The updated version should show up at
https://www.python.org/dev/peps/pep-0557/ shortly.

The major changes from the previous version are:

- Add InitVar to specify initialize-only fields.
- Renamed __dataclass_post_init__() to __post_init().
- Rename cmp to compare.
- Added eq, separate from compare, so you can test
   unorderable items for equality.
- Flushed out asdict() and astuple().
- Changed replace() to just call __init__(), and dropped
   the complex post-create logic.

The only open issues I know of are:
- Should object comparison require an exact match on the type?
   https://github.com/ericvsmith/dataclasses/issues/51
- Should the replace() function be renamed to something else?
   https://github.com/ericvsmith/dataclasses/issues/77

Most of the items that were previously discussed on python-dev were
discussed in detail at https://github.com/ericvsmith/dataclasses. Before
rehashing an old discussion, please check there first.

Also at https://github.com/ericvsmith/dataclasses is an implementation,
with tests, that should work with 3.6 and 3.7. The only action item for
the code is to clean up the implementation of InitVar, but that's
waiting for PEP 560. Oh, and if PEP 563 is accepted I'll also need to do
some work.

Feedback is welcomed!

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/carl%40oddbird.net




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/26/2017 3:35 AM, Eric V. Smith wrote:

On 11/26/2017 12:23 AM, Carl Meyer wrote:

A couple minor
questions:


If compare is True, then eq is ignored, and __eq__ and __ne__ will be

automatically generated.

IMO it's generally preferable to make nonsensical parameter combinations
an immediate error, rather than silently ignore one of them. Is there a
strong reason for letting nonsense pass silently here?

(I reviewed the previous thread; there was a lot of discussion about
enums/flags vs two boolean params, but I didn't see explicit discussion
of this issue; the only passing references I noticed said the invalid
combo should be "disallowed", e.g. Guido in [1], which to me implies "an
error.")


I think you're right here. I'll change it to a ValueError.


While creating an issue for this 
(https://github.com/ericvsmith/dataclasses/issues/88), it occurs to me 
that the class-level parameter really should be "order" or "orderable", 
not "compare". It made more sense when it was called "cmp", but 
"compare" now seems wrong.


Because "eq" says "can I compare two instances", and what's currently 
called "compare" is "can I order two instances". Nick had a similar 
suggestion before the PEP was written 
(https://mail.python.org/pipermail/python-ideas/2017-May/045740.html).


The field-level parameter should stay "compare", because it's used for 
both __gt__ and friends, as well as __eq__ and __ne__. It's saying "is 
this field used in all of the comparison methods".


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/26/2017 3:48 AM, Eric V. Smith wrote:
While creating an issue for this 
(https://github.com/ericvsmith/dataclasses/issues/88), it occurs to me 
that the class-level parameter really should be "order" or "orderable", 
not "compare". It made more sense when it was called "cmp", but 
"compare" now seems wrong.


Because "eq" says "can I compare two instances", and what's currently 
called "compare" is "can I order two instances". Nick had a similar 
suggestion before the PEP was written 
(https://mail.python.org/pipermail/python-ideas/2017-May/045740.html).


The field-level parameter should stay "compare", because it's used for 
both __gt__ and friends, as well as __eq__ and __ne__. It's saying "is 
this field used in all of the comparison methods".


I created https://github.com/ericvsmith/dataclasses/issues/90 for this. 
I think I'll leave 'eq' alone, and change 'compare' to 'order', for the 
class-level parameter name.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/26/2017 1:04 PM, Brett Cannon wrote:

The only open issues I know of are:
- Should object comparison require an exact match on the type?
https://github.com/ericvsmith/dataclasses/issues/51


I say don't require the type comparison for duck typing purposes.


The problem with that is that you end up with cases like this, which I 
don't think we want:


@dataclass
class Point:
x: int
y: int

@dataclass
class Point3d:
x: int
y: int
z: int

assert Point(1, 2) == Point3d(1, 2, 3)

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/27/2017 1:04 AM, Nick Coghlan wrote:

On 27 November 2017 at 15:04, Greg Ewing  wrote:

Nick Coghlan wrote:


Perhaps the check could be:

   (type(lhs) == type(rhs) or fields(lhs) == fields(rhs)) and all
(individual fields match)



I think the types should *always* have to match, or at least
one should be a subclass of the other. Consider:

@dataclass
class Point3d:
 x: float
 y: float
 z: float

@dataclass
class Vector3d:
 x: float
 y: float
 z: float

Points and vectors are different things, and they should never
compare equal, even if they have the same field names and values.


And I guess if folks actually want more permissive structure-based
matching, that's one of the features that collections.namedtuple
offers that data classes don't.


And in this case you could also do:
astuple(point) == astuple(vector)

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/27/2017 6:01 AM, Sebastian Rittau wrote:

On 25.11.2017 22:06, Eric V. Smith wrote:



The major changes from the previous version are:

- Add InitVar to specify initialize-only fields. 


This is the only feature that does not sit right with me. It looks very 
obscure and "hacky". From what I understand, we are supposed to use the 
field syntax to define constructor arguments. I'd argue that the name 
"initialize-only fields" is a misnomer, which only hides the fact that 
this has nothing to do with fields at all. Couldn't dataclassses just 
pass *args and **kwargs to __post_init__()? Type checkers need to be 
special-cases for InitVar anyway, couldn't they instead be special cased 
to look at __post_init__ argument types?


First off, I expect this feature to be used extremely rarely. I'm 
tempted to remove it, since it's infrequently needed and it could be 
added later. And as the PEP points out, you can get most of the way with 
an alternate classmethod constructor.


I had something like your suggestion half coded up, except I inspected 
the args to __post_init__() and added them to __init__, avoiding the 
API-unfriendly *args and **kwargs.


So in:
@dataclass
class C:
x: int
y: int

def __post_init__(self, database: DatabaseType): pass

Then the __init__ signature became:

def __init__(self, x:int, y:int, database:DatabaseType):

In the end, that seems like a lot of magic (but what about this isn't?), 
it required the inspect module to be imported, and I thought it made 
more sense for all of the init params to be near each other:


@dataclass
class C:
x: int
y: int
database: InitVar[DatabaseType]

def __post_init__(self, database): pass

No matter what we do here, static type checkers are going to have to be 
aware of either the InitVars or the hoisting of params from 
__post_init__ to __init__.


One other thing about InitVar: it lets you control where the init-only 
parameter goes in the __init__ call. This is especially important with 
default values:


@dataclass
class C:
x: int
database: InitVar[DatabaseType]
y: int = 0

def __post_init__(self, database): pass

In this case, if I were hoisting params from __post_init__ to __init__, 
the __init__ call would be:


def __init__(self, x, y=0, database)

Which is an error. I guess you could say the init-only parameters would 
go first in the __init__ definition, but then you have the same problem 
if any of them have default values.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/27/2017 7:31 AM, Sebastian Rittau wrote:

On 27.11.2017 13:23, Eric V. Smith wrote:
I had something like your suggestion half coded up, except I inspected 
the args to __post_init__() and added them to __init__, avoiding the 
API-unfriendly *args and **kwargs.
I understand your concerns with *args and **kwargs. I think we need to 
find a solution for that eventually.


One other thing about InitVar: it lets you control where the init-only 
parameter goes in the __init__ call. This is especially important with 
default values:


This is indeed a nice property. I was thinking about that myself and how 
to best handle it. One use case that could occur in out codebase is 
passing in a "context" argument. By convention, this is always the first 
argument to the constructor, so it would be nice if this would also work 
for dataclasses.


And that's the one thing that you can't do with an alternate classmethod 
constructor, and is the reason I added InitVar: you can't force a 
non-field parameter such as a context (or in my example, a database) to 
be always present when instances are constructed. And also consider the 
"replace()" module method. InitVars must also be supplied there, whereas 
with a classmethod constructor, they wouldn't be.


This is for the case where a context or database is needed to construct 
the instance, but isn't stored as a field on the instance. Again, not 
super-common, but it does happen. My point here is not that InitVar is 
better than __post_init__ parameter hoisting for this specific need, but 
that both of them provide something that classmethod constructors do 
not. I'll add some wording on this to the PEP.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/27/2017 7:26 AM, Sebastian Rittau wrote:

On 27.11.2017 12:01, Sebastian Rittau wrote:



The major changes from the previous version are:

- Add InitVar to specify initialize-only fields. 


This is the only feature that does not sit right with me. It looks 
very obscure and "hacky". From what I understand, we are supposed to 
use the field syntax to define constructor arguments. I'd argue that 
the name "initialize-only fields" is a misnomer, which only hides the 
fact that this has nothing to do with fields at all. Couldn't 
dataclassses just pass *args and **kwargs to __post_init__()? Type 
checkers need to be special-cases for InitVar anyway, couldn't they 
instead be special cased to look at __post_init__ argument types? 
I am sorry for the double post, but I thought a bit more about why this 
does not right with me:


  * As written above, InitVars look like fields, but aren't.


Same as with ClassVars, which is where the inspiration came from.


  * InitVar goes against the established way to pass through arguments,
*args and **kwargs. While type checking those is an unsolved
problem, from what I understand, I don't think we should introduce a
second way just for dataclasses.
  * InitVars look like a way to satisfy the type checker without
providing any benefit to the programmer. Even when I'm not
interested in type checking, I have to declare init vars.


Same as with ClassVars, if you're using them. And that's not just a 
dataclasses thing, although dataclasses is the first place I know of 
where it would change the code semantics.



  * InitVars force me to repeat myself. I have the InitVar declaration
and then I have the repeat myself in the signature of
__post_init__(). This has all the usual problems of repeated code.


There was some discussion about this starting at 
https://github.com/ericvsmith/dataclasses/issues/17#issuecomment-345529717, 
in particular a few messages down where we discussed what would be 
repeated, and what mypy would be able to deduce. You won't need to 
repeat the type declaration.



I hope I did not misunderstood the purpose of InitVar.


I think you understand it perfectly well, especially with the "context" 
discussion. Thanks for bringing it up.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/27/17 10:51 AM, Guido van Rossum wrote:

Following up on this subthread (inline below).

On Mon, Nov 27, 2017 at 2:56 AM, Eric V. Smith mailto:e...@trueblade.com>> wrote:

On 11/27/2017 1:04 AM, Nick Coghlan wrote:

On 27 November 2017 at 15:04, Greg Ewing
mailto:greg.ew...@canterbury.ac.nz>> wrote:

Nick Coghlan wrote:


Perhaps the check could be:

   (type(lhs) == type(rhs) or fields(lhs) ==
fields(rhs)) and all
(individual fields match)



I think the types should *always* have to match, or at least
one should be a subclass of the other. Consider:

@dataclass
class Point3d:
 x: float
 y: float
 z: float

@dataclass
class Vector3d:
 x: float
 y: float
 z: float

Points and vectors are different things, and they should never
compare equal, even if they have the same field names and
values.


And I guess if folks actually want more permissive structure-based
matching, that's one of the features that collections.namedtuple
offers that data classes don't.


And in this case you could also do:
astuple(point) == astuple(vector)


Didn't we at one point have something like

isinstance(other, self.__class__) and fields(other) == fields(self) and


(plus some optimization if the types are identical)?

That feels ideal, because it means you can subclass Point just to add
some methods and it will stay comparable, but if you add fields it will
always be unequal.


I don't think we had that before, but it sounds right to me. I think it 
could be:


isinstance(other, self.__class__) and len(fields(other)) == 
len(fields(self)) and 


Since by definition if you're a subclass you'll start with all of the 
same fields. So if the len's match, you won't have added any new fields. 
That should be sufficiently cheap.


Then the optimized version would be:

(self.__class__ is other.__class__) or (isinstance(other, 
self.__class__) and len(fields(other)) == len(fields(self))) and individual fields match>


I'd probably further optimize len(fields(obj)), but that's the general idea.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/27/2017 10:51 AM, Guido van Rossum wrote:

Following up on this subthread (inline below).



Didn't we at one point have something like

isinstance(other, self.__class__) and fields(other) == fields(self) and 



(plus some optimization if the types are identical)?

That feels ideal, because it means you can subclass Point just to add 
some methods and it will stay comparable, but if you add fields it will 
always be unequal.


One thing this doesn't let you do is compare instances of two different 
subclasses of a base type:


@dataclass
class B:
i: int

@dataclass
class C1(B): pass

@dataclass
class C2(B): pass

You can't compare C1(0) and C2(0), because neither one is an instance of 
the other's type. The test to get this case to work would be expensive: 
find the common ancestor, and then make sure no fields have been added 
since then. And I haven't thought through multiple inheritance.


I suggest we don't try to support this case.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/28/17 7:02 AM, Nick Coghlan wrote:

On 28 November 2017 at 17:41, Eric V. Smith  wrote:

One thing this doesn't let you do is compare instances of two different
subclasses of a base type:

@dataclass
class B:
i: int

@dataclass
class C1(B): pass

@dataclass
class C2(B): pass

You can't compare C1(0) and C2(0), because neither one is an instance of the
other's type. The test to get this case to work would be expensive: find the
common ancestor, and then make sure no fields have been added since then.
And I haven't thought through multiple inheritance.

I suggest we don't try to support this case.


That gets you onto problematic ground as far as transitivity is
concerned, since you'd end up with the following:


Excellent point, thanks for raising it.


>>> b = B(0); c1 = C1(0); c2 = C2(0)
>>> c1 == b
True
>>> b == c2
True
>>> c1 == c2
False

However, I think you can fix this by injecting the first base in the
MRO that defines a data field as a "__field_layout__" class attribute,
and then have the comparison methods check for "other.__field_layout__
is self.__field_layout__", rather than checking the runtime class
directly.

So in the above example, you would have:

   >>> B.__field_layout__ is B
   True
   >>> C1.__field_layout__ is B
   True
   >>> C2.__field_layout__ is B
   True

It would then be up to the dataclass decorator to set
`__field_layout__` correctly, using the follow rules:

1. Use the just-defined class if the class defines any fields
2. Use the just-defined class if it inherits from multiple base
classes that define fields and don't already share an MRO
3. Use a base class if that's either the only base class that defines
fields, or if all other base classes that define fields are already in
the MRO of that base class


That seems like a lot of complication for a feature that will be rarely 
used. I'll give it some thought, especially the MI logic.


I think what you're laying out is an optimization for "do the classes 
have identical fields, inherited through a common base class or 
classes", right?


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/28/2017 1:57 PM, Guido van Rossum wrote:
I would also be happy with a retreat, where we define `__eq__` to insist 
that the classes are the same, and people can override this to their 
hearts' content.


I agree. And I guess we could always add it later, if there's a huge 
demand and someone writes a decent implementation. There would be a 
slight backwards incompatibility, though. Frankly, I think it would 
never be needed.


One question remains: do we do what attrs does: for `__eq__` and 
`__ne__` use an exact type match, and for the 4 ordered comparison 
operators use an isinstance check? On the comparison operators, they 
also ignore attributes defined on any derived class [0]. As I said, I 
couldn't find an attrs issue that discusses their choice. I'll ask Hynek 
over on the dataclasses github issue.


Currently the dataclasses code on master uses an exact type match for 
all 6 methods.


Eric.

[0] That is, they do the following (using dataclasses terms):

Given:

@dataclass
class B:
i: int
j: int

@dataclass
class C:
k: int

Then B.__eq__ is:

def __eq__(self, other):
if other.__class__ is self.__class__:
return (other.i, other.j) == (self.i, self.j)
return NotImplemented

And B.__lt__ is:

def __lt__(self, other):
if isinstance(other, self.__class__):
return (other.i, other.j) < (self.i, self.j)
return NotImplemented

So if you do:
b = B(1, 2)
c = C(1, 2, 3)

Then `B(1, 2) < C(1, 2, 3)` ignores `c.k`.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/28/2017 4:14 PM, Guido van Rossum wrote:
Hm. Maybe for the ordering comparisons we could defer to the class with 
the longest list of fields, as long as there's a subtype relationship? 
That way bb would be equivalent, and both would use C.__gt__. 
Which had better not reject this on the basis that other is not an 
instance of a subclass of C.


IIRC there's already something in the interpreter that tries the most 
derived class first for binary operators -- that may force our hand here.


I'm leaning toward doing the same thing attrs does. They have much more 
experience with this.


This is my last open specification issue. I think I'm ready for a 
pronouncement on the PEP once I do one final editing pass, in hopes of 
getting this in 3.7.0a3 by next weekend. If the comparisons need to 
change, I'm okay with doing that in the beta.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Second post: PEP 557, Data Classes

[Eric V. Smith]

On 11/28/2017 8:31 PM, Eric V. Smith wrote:

On 11/28/2017 4:14 PM, Guido van Rossum wrote:
Hm. Maybe for the ordering comparisons we could defer to the class 
with the longest list of fields, as long as there's a subtype 
relationship? That way bb would be equivalent, and both would 
use C.__gt__. Which had better not reject this on the basis that other 
is not an instance of a subclass of C.


IIRC there's already something in the interpreter that tries the most 
derived class first for binary operators -- that may force our hand here.


I'm leaning toward doing the same thing attrs does. They have much more 
experience with this.


Except that given Hynek's response in 
https://github.com/ericvsmith/dataclasses/issues/51#issuecomment-347769322, 
I'm just going to leave it as-is, with a strict type requirement for all 
6 methods.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the status of PEP 505: None-aware operators?

[Eric V. Smith]

> On Nov 29, 2017, at 12:40 PM, David Mertz  wrote:
> 
> I like much of the thinking in Random's approach. But I still think None 
> isn't quite special enough to warrant it's own syntax.
> 
> However, his '(or None: name.strip()[4:].upper())' makes me realize that what 
> is being asked in all the '?(', '?.', '?[' syntax ideas is a kind of ternary 
> expression.  Except the ternary isn't based on whether a predicate holds, but 
> rather on whether an exception occurs (AttributeError, KeyError, TypeError).  
> And the fallback in the ternary is always None rather than being general.
> 
> I think we could generalize this to get something both more Pythonic and more 
> flexible.  E.g.:
> 
> val = name.strip()[4:].upper() except None
> 
> This would just be catching all errors, which is perhaps too broad.  But it 
> *would* allow a fallback other than None:
> 
> val = name.strip()[4:].upper() except -1
> 
> I think some syntax could be possible to only "catch" some exceptions and let 
> others propagate.  Maybe:
> 
> val = name.strip()[4:].upper() except (AttributeError, KeyError): -1
> 
> I don't really like throwing a colon in an expression though.  Perhaps some 
> other word or symbol could work instead.  How does this read:
> 
> val = name.strip()[4:].upper() except -1 in (AttributeError, KeyError)
> 
> Where the 'in' clause at the end would be optional, and default to 
> 'Exception'.
> 
> I'll note that what this idea DOES NOT get us is:
> 
>   val = timeout ?? local_timeout ?? global_timeout
> 
> Those values that are "possibly None" don't raise exceptions, so they 
> wouldn't apply to this syntax.

See the rejected PEP 463 for Exception catching expressions. 

Eric. 

> 
> Yours, David...
> 
> 
>> On Wed, Nov 29, 2017 at 9:03 AM, Random832  wrote:
>> On Tue, Nov 28, 2017, at 15:31, Raymond Hettinger wrote:
>> >
>> > > I also cc python-dev to see if anybody here is strongly in favor or 
>> > > against this inclusion.
>> >
>> > Put me down for a strong -1.   The proposal would occasionally save a few
>> > keystokes but comes at the expense of giving Python a more Perlish look
>> > and a more arcane feel.
>> >
>> > One of the things I like about Python is that I can walk non-programmers
>> > through the code and explain what it does.  The examples in PEP 505 look
>> > like a step in the wrong direction.  They don't "look like Python" and
>> > make me feel like I have to decrypt the code to figure-out what it does.
>> >
>> > timeout ?? local_timeout ?? global_timeout
>> > 'foo' in (None ?? ['foo', 'bar'])
>> > requested_quantity ?? default_quantity * price
>> > name?.strip()[4:].upper()
>> > user?.first_name.upper()
>> 
>> Since we're looking at different syntax for the ?? operator, I have a
>> suggestion for the ?. operator - and related ?[] and ?() that appeared
>> in some of the proposals. How about this approach?
>> 
>> Something like (or None: ...) as a syntax block in which any operation
>> [lexically within the expression, not within e.g. called functions, so
>> it's different from simply catching AttributeError etc, even if that
>> could be limited to only catching when the operand is None] on None that
>> is not valid for None will yield None instead.
>> 
>> This isn't *entirely* equivalent, but offers finer control.
>> 
>> v = name?.strip()[4:].upper() under the old proposal would be more or
>> less equivalent to:
>> 
>> v = name.strip()[4:].upper() if name is not None else None
>> 
>> Whereas, you could get the same result with:
>> (or None: name.strip()[4:].upper())
>> 
>> Though that would technically be equivalent to these steps:
>> v = name.strip if name is not None else None
>> v = v() if v "
>> v = v[4:] """
>> v = v.upper """
>> v = v() """
>> 
>> The compiler could optimize this case since it knows none of the
>> operations are valid on None. This has the advantage of being explicit
>> about what scope the modified rules apply to, rather than simply
>> implicitly being "to the end of the chain of dot/bracket/call operators"
>> 
>> It could also be extended to apply, without any additional syntax, to
>> binary operators (result is None if either operand is None) (or None: a
>> + b), for example, could return None if either a or b is none.
>> 
>> [I think I proposed this before with the syntax ?(...), the (or None:
>> ...) is just an idea to make it look more like Python.]
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: 
>> https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx
> 
> 
> 
> -- 
> Keeping medicines from the bloodstreams of the sick; food 
> from the bellies of the hungry; books from the hands of the 
> uneducated; technology from the underdeveloped; and putting 
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
> ___

[Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

I've posted a new version of PEP 557, it should soon be available at 
https://www.python.org/dev/peps/pep-0557/.


The only significant changes since the last version are:

- changing the "compare" parameter to be "order", since that more 
accurately reflects what it does.
- Having the combination of "eq=False" and "order=True" raise an 
exception instead of silently changing eq to True.


There were no other issues raised with the previous version of the PEP.

So with that, I think it's ready for a pronouncement.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

On 11/29/2017 6:51 PM, Carl Meyer wrote:

On 11/29/2017 03:26 PM, Eric V. Smith wrote:

I've posted a new version of PEP 557, it should soon be available at
https://www.python.org/dev/peps/pep-0557/.

The only significant changes since the last version are:

- changing the "compare" parameter to be "order", since that more
accurately reflects what it does.
- Having the combination of "eq=False" and "order=True" raise an
exception instead of silently changing eq to True.

There were no other issues raised with the previous version of the PEP.


Not quite; I also raised the issue of isdataclass(ADataClass) returning
False. I still think that's likely to be a cause of bug reports if left
as-is.


Oops, sorry about that!

I think you're probably right. attr.has(), which is the equivalent attrs 
API, returns True for both classes and instances (although 
interestingly, the code only talks about it working on classes).


https://github.com/ericvsmith/dataclasses/issues/99

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

On 11/29/2017 6:26 PM, Eric V. Smith wrote:
I've posted a new version of PEP 557, it should soon be available at 
https://www.python.org/dev/peps/pep-0557/.


Also, I've posted version 0.2 to PyPI as dataclasses, so you can play 
with it on 3.6 and 3.7.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

On 11/30/2017 6:59 AM, Antoine Pitrou wrote:


Or, simply, is_dataclass_instance(), which is even longer, but far more
readable thanks to explicit word boundaries :-)


That actually doesn't bother me. I think this API will be used rarely, 
if ever. Or more realistically, it should be used rarely: what actually 
happens will no doubt surprise me.


So I'm okay with is_dataclass_instance() and is_dataclass_class().

But then I'm also okay with dropping the API entirely. nametuple has 
lived for years without it, although Raymond's advice there is that if 
you really want to know, look for _fields. See 
https://bugs.python.org/issue7796#msg99869 and the following discussion.


Eric.



On Thu, 30 Nov 2017 10:16:58 +0100
Antoine Pitrou  wrote:

isdataclass() testing for instance-ship does sound like a bug magnet to
me.

If isdataclassinstance() is too long (that's understandable), how about
isdatainstance()?

Regards

Antoine.


On Wed, 29 Nov 2017 17:02:21 -0800
Guido van Rossum  wrote:

  On Wed, Nov 29, 2017 at 3:51 PM, Carl Meyer  wrote:
   

On 11/29/2017 03:26 PM, Eric V. Smith wrote:

I've posted a new version of PEP 557, it should soon be available at
https://www.python.org/dev/peps/pep-0557/.

The only significant changes since the last version are:

- changing the "compare" parameter to be "order", since that more
accurately reflects what it does.
- Having the combination of "eq=False" and "order=True" raise an
exception instead of silently changing eq to True.

There were no other issues raised with the previous version of the PEP.


Not quite; I also raised the issue of isdataclass(ADataClass) returning
False. I still think that's likely to be a cause of bug reports if left
as-is.



I tried to look up the discussion but didn't find much except that you
flagged this as an issue. To repeat, your concern is that isdataclass()
applies to *instances*, not classes, which is how Eric has designed it, but
you worry that either through the name or just because people don't read
the docs it will be confusing. What do you suppose we do? I think making it
work for classes as well as for instances would cause another category of
bugs (confusion between cases where a class is needed vs. an instance
abound in other situations -- we don't want to add to that). Maybe it
should raise TypeError when passed a class (unless its metaclass is a
dataclass)? Maybe it should be renamed to isdataclassinstance()? That's a
mouthful, but I don't know how common the need to call this is, and people
who call it a lot can define their own shorter alias.
   








___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

On 11/30/2017 1:30 PM, Brett Cannon wrote:



On Thu, 30 Nov 2017 at 05:00 Eric V. Smith <mailto:e...@trueblade.com>> wrote:


On 11/30/2017 6:59 AM, Antoine Pitrou wrote:
 >
 > Or, simply, is_dataclass_instance(), which is even longer, but
far more
 > readable thanks to explicit word boundaries :-)

That actually doesn't bother me. I think this API will be used rarely,
if ever. Or more realistically, it should be used rarely: what actually
happens will no doubt surprise me.

So I'm okay with is_dataclass_instance() and is_dataclass_class().

But then I'm also okay with dropping the API entirely. nametuple has
lived for years without it, although Raymond's advice there is that if
you really want to know, look for _fields. See
https://bugs.python.org/issue7796#msg99869 and the following discussion.


My question was going to be whether this is even necessary. :) Perhaps 
we just drop it for now and add it in if we find there's a public need 
for it?


That's what I'm leaning toward. I've been trying to figure out what 
attr.has() or hasattr(obj, '_fields') are actually used for. The attrs 
version is hard to search for, and while I see the question about 
namedtuples asked fairly often on SO, I haven't seen an actual use case.


It's easy enough for someone to write their own isdataclass(), 
admittedly using an undocumented feature.


So I'm thinking let's drop it and then gauge the demand for it, if any.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

On 11/30/2017 7:22 PM, Eric V. Smith wrote:

On 11/30/2017 1:30 PM, Brett Cannon wrote:



On Thu, 30 Nov 2017 at 05:00 Eric V. Smith <mailto:e...@trueblade.com>> wrote:


    On 11/30/2017 6:59 AM, Antoine Pitrou wrote:
 >
 > Or, simply, is_dataclass_instance(), which is even longer, but
    far more
 > readable thanks to explicit word boundaries :-)

    That actually doesn't bother me. I think this API will be used 
rarely,
    if ever. Or more realistically, it should be used rarely: what 
actually

    happens will no doubt surprise me.

    So I'm okay with is_dataclass_instance() and is_dataclass_class().

    But then I'm also okay with dropping the API entirely. nametuple has
    lived for years without it, although Raymond's advice there is 
that if

    you really want to know, look for _fields. See
    https://bugs.python.org/issue7796#msg99869 and the following 
discussion.



My question was going to be whether this is even necessary. :) Perhaps 
we just drop it for now and add it in if we find there's a public need 
for it?


That's what I'm leaning toward. I've been trying to figure out what 
attr.has() or hasattr(obj, '_fields') are actually used for. The attrs 
version is hard to search for, and while I see the question about 
namedtuples asked fairly often on SO, I haven't seen an actual use case.


It's easy enough for someone to write their own isdataclass(), 
admittedly using an undocumented feature.


Actually there's a supported way to write your own isdataclass(): call 
dataclasses.fields(obj). If it throws a TypeError, it's not a dataclass 
instance or class. I'll add a note to the PEP.


Eric.



So I'm thinking let's drop it and then gauge the demand for it, if any.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

On 11/30/2017 3:35 PM, Carl Meyer wrote:

On 11/29/2017 05:02 PM, Guido van Rossum wrote:

I tried to look up the discussion but didn't find much except that you
flagged this as an issue. To repeat, your concern is that isdataclass()
applies to *instances*, not classes, which is how Eric has designed it,
but you worry that either through the name or just because people don't
read the docs it will be confusing. What do you suppose we do? I think
making it work for classes as well as for instances would cause another
category of bugs (confusion between cases where a class is needed vs. an
instance abound in other situations -- we don't want to add to that).
Maybe it should raise TypeError when passed a class (unless its
metaclass is a dataclass)? Maybe it should be renamed to
isdataclassinstance()? That's a mouthful, but I don't know how common
the need to call this is, and people who call it a lot can define their
own shorter alias.


Yeah, I didn't propose a specific fix because I think there are several
options (all mentioned in this thread already), and I don't really have
strong feelings about them:

1) Keep the existing function and name, let it handle either classes or
instances. I agree that this is probably not the best option available,
though IMO it's still marginally better than the status quo).

2) Punt the problem by removing the function; don't add it to the public
API at all until we have demonstrated demand.

3) Rename it to "is_dataclass_instance" (and maybe also keep a separate
"is_dataclass" for testing classes directly). (Then there's also the
choice about raising TypeError vs just returning False if a function is
given the wrong type; I think TypeError is better.)


In that case, you can spell "is_dataclass_instance":

def isdataclass_instance(obj):
dataclasses.fields(obj)  # raises TypeError for non-dataclass
 # classes or instances
if not isinstance(obj, type):
raise TypeError('not an instance')
return True

Since this is easy enough to do in your own code, and I still don't see 
a use case, I'll just add a note to the PEP and omit delete isdataclass().


Plus, you can decide for yourself how to deal with the question of 
returning true for classes or instances or both.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

Since this is easy enough to do in your own code, and I still don't see 
a use case, I'll just add a note to the PEP and delete isdataclass().


Plus, you can decide for yourself how to deal with the question of 
returning true for classes or instances or both.


I've updated the PEP and reposted it. The only change is removing 
isdataclass().


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Third and hopefully final post: PEP 557, Data Classes

[Eric V. Smith]

See https://github.com/ericvsmith/dataclasses/issues/104 for a 
discussion on making order=False the default. This matches regular 
classes in Python 3, which cannot be ordered.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

I've pushed another version of PEP 557. The only difference is changing 
the default value of "order" to False instead of True. This matches 
regular classes: instances can be tested for equality, but are unordered.


Discussion at https://github.com/ericvsmith/dataclasses/issues/104

It's already available at https://www.python.org/dev/peps/pep-0557/

I've updated the implementation on PyPI to reflect this change: 
https://pypi.python.org/pypi/dataclasses/0.3


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

I've made a minor change: the return type of fields() is now a tuple, it 
was a list.


Eric.

On 12/2/2017 9:02 AM, Eric V. Smith wrote:
I've pushed another version of PEP 557. The only difference is changing 
the default value of "order" to False instead of True. This matches 
regular classes: instances can be tested for equality, but are unordered.


Discussion at https://github.com/ericvsmith/dataclasses/issues/104

It's already available at https://www.python.org/dev/peps/pep-0557/

I've updated the implementation on PyPI to reflect this change: 
https://pypi.python.org/pypi/dataclasses/0.3


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

On 12/3/2017 11:56 AM, Guido van Rossum wrote:
Not sure I like that better. It's an open-ended sequence of homogeneous 
types. What's the advantage of a tuple? I don't want to blindly follow 
existing APIs.


So people don't modify it, but consenting adults would say "don't do 
that". I currently return a new tuple in each call to fields(), but in 
the future I might return the same one every time (per class).


I really don't care so much. The only reason I made any change was 
because the implementation was returning an OrderedDict, so I was 
changing the tests anyway. I'm happy to change it back to a list, based 
on the convention of homogeneous types being in a list.


Eric.



On Sun, Dec 3, 2017 at 6:55 AM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


I've made a minor change: the return type of fields() is now a
tuple, it was a list.

Eric.

On 12/2/2017 9:02 AM, Eric V. Smith wrote:

I've pushed another version of PEP 557. The only difference is
changing the default value of "order" to False instead of True.
This matches regular classes: instances can be tested for
equality, but are unordered.

Discussion at
https://github.com/ericvsmith/dataclasses/issues/104
<https://github.com/ericvsmith/dataclasses/issues/104>

It's already available at
https://www.python.org/dev/peps/pep-0557/
<https://www.python.org/dev/peps/pep-0557/>

I've updated the implementation on PyPI to reflect this change:
https://pypi.python.org/pypi/dataclasses/0.3
<https://pypi.python.org/pypi/dataclasses/0.3>

Eric.
___
Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>
Unsubscribe:

https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

<https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com>



___
Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org 
<https://mail.python.org/mailman/options/python-dev/guido%40python.org>




--
--Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

Me, either. So I'm going to leave it as a tuple. Unless I find something 
while reviewing it tonight, I'm done.


Eric.

On 12/3/2017 3:02 PM, Guido van Rossum wrote:

On second thought I don't care that much.

On Dec 3, 2017 9:07 AM, "Eric V. Smith" <mailto:e...@trueblade.com>> wrote:


On 12/3/2017 11:56 AM, Guido van Rossum wrote:

Not sure I like that better. It's an open-ended sequence of
homogeneous types. What's the advantage of a tuple? I don't want
to blindly follow existing APIs.


So people don't modify it, but consenting adults would say "don't do
that". I currently return a new tuple in each call to fields(), but
in the future I might return the same one every time (per class).

I really don't care so much. The only reason I made any change was
because the implementation was returning an OrderedDict, so I was
changing the tests anyway. I'm happy to change it back to a list,
based on the convention of homogeneous types being in a list.

Eric.


On Sun, Dec 3, 2017 at 6:55 AM, Eric V. Smith
mailto:e...@trueblade.com>
<mailto:e...@trueblade.com <mailto:e...@trueblade.com>>> wrote:

     I've made a minor change: the return type of fields() is now a
         tuple, it was a list.

     Eric.

     On 12/2/2017 9:02 AM, Eric V. Smith wrote:

         I've pushed another version of PEP 557. The only
difference is
         changing the default value of "order" to False instead
of True.
         This matches regular classes: instances can be tested for
         equality, but are unordered.

         Discussion at
https://github.com/ericvsmith/dataclasses/issues/104
<https://github.com/ericvsmith/dataclasses/issues/104>
         <https://github.com/ericvsmith/dataclasses/issues/104
<https://github.com/ericvsmith/dataclasses/issues/104>>

         It's already available at
https://www.python.org/dev/peps/pep-0557/
<https://www.python.org/dev/peps/pep-0557/>
         <https://www.python.org/dev/peps/pep-0557/
<https://www.python.org/dev/peps/pep-0557/>>

         I've updated the implementation on PyPI to reflect this
change:
https://pypi.python.org/pypi/dataclasses/0.3
<https://pypi.python.org/pypi/dataclasses/0.3>
         <https://pypi.python.org/pypi/dataclasses/0.3
<https://pypi.python.org/pypi/dataclasses/0.3>>

         Eric.
         ___
         Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
<mailto:Python-Dev@python.org <mailto:Python-Dev@python.org>>
https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>
         <https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>>
         Unsubscribe:

https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

<https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com>

<https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


<https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com>>



     ___
     Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
<mailto:Python-Dev@python.org <mailto:Python-Dev@python.org>>
https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>
     <https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>>
     Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
<https://mail.python.org/mailman/options/python-dev/guido%40python.org>
<https://mail.python.org/mailman/options/python-dev/guido%40python.org
<https://mail.python.org/mailman/options/python-dev/guido%40python.org>>




-- 
--Guido van Rossum (python.org/~guido

<http://python.org/%7Eguido> <http://python.org/%7Eguido>)




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

On 12/3/2017 3:33 PM, Antoine Pitrou wrote:

On Sat, 2 Dec 2017 09:02:37 -0500
"Eric V. Smith"  wrote:

I've pushed another version of PEP 557. The only difference is changing
the default value of "order" to False instead of True. This matches
regular classes: instances can be tested for equality, but are unordered.

Discussion at https://github.com/ericvsmith/dataclasses/issues/104

It's already available at https://www.python.org/dev/peps/pep-0557/


Thanks.  I have to ask: why don't "asdict" and "astuple" respect PEP 8
naming?


I guess it depends if you think the underscore is needed to improve 
readability. "Function names should be lowercase, with words separated 
by underscores as necessary to improve readability."


I don't feel strongly enough about it to change it, but part of that is 
because I'm burned out on the PEP, so I might not be a good one to judge 
at this point. I guess if I clear my head and I were doing it from 
scratch again I'd make them as_dict and as_tuple, so maybe I should 
brush aside inertia and make the change.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

On 12/3/2017 8:31 PM, Guido van Rossum wrote:
On Sun, Dec 3, 2017 at 1:28 PM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


On 12/3/2017 3:33 PM, Antoine Pitrou wrote:





Thanks.  I have to ask: why don't "asdict" and "astuple" respect
PEP 8
naming?


I guess it depends if you think the underscore is needed to improve
readability. "Function names should be lowercase, with words
separated by underscores as necessary to improve readability."

I don't feel strongly enough about it to change it, but part of that
is because I'm burned out on the PEP, so I might not be a good one
to judge at this point. I guess if I clear my head and I were doing
it from scratch again I'd make them as_dict and as_tuple, so maybe I
should brush aside inertia and make the change.


The Python stdlib is incredibly inconsistent where it comes to inserting 
underscores. I think in this case it matches `namedtuple._asdict()` and 
that's good enough for me.


It also matches `attrs.asdict()`, which is what originally inspired it.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

On 12/3/2017 9:07 PM, Eric V. Smith wrote:


It also matches `attrs.asdict()`, which is what originally inspired it.


Make that `attr.asdict()`. So easy to get that wrong.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

On 12/3/2017 9:07 PM, Eric V. Smith wrote:

On 12/3/2017 8:31 PM, Guido van Rossum wrote:
On Sun, Dec 3, 2017 at 1:28 PM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


    On 12/3/2017 3:33 PM, Antoine Pitrou wrote:





    Thanks.  I have to ask: why don't "asdict" and "astuple" respect
    PEP 8
    naming?


    I guess it depends if you think the underscore is needed to improve
    readability. "Function names should be lowercase, with words
    separated by underscores as necessary to improve readability."

    I don't feel strongly enough about it to change it, but part of that
    is because I'm burned out on the PEP, so I might not be a good one
    to judge at this point. I guess if I clear my head and I were doing
    it from scratch again I'd make them as_dict and as_tuple, so maybe I
    should brush aside inertia and make the change.


The Python stdlib is incredibly inconsistent where it comes to 
inserting underscores. I think in this case it matches 
`namedtuple._asdict()` and that's good enough for me.


It also matches `attrs.asdict()`, which is what originally inspired it.


After a brief discussion at 
https://github.com/ericvsmith/dataclasses/issues/110, the decision is to 
leave the function names as-is, without underscores, to be consistent 
with namedtuples and attrs.


I'll add a note in the PEP's discussion section.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

Thanks, Guido. And indeed, thanks to everyone else who provided inspiration and 
feedback. I too would like to thank Hynek and the other authors of “attrs”. 

I’ll get the implementation committed in the next day or so. 

--
Eric.

> On Dec 4, 2017, at 12:17 PM, Guido van Rossum  wrote:
> 
> And with this, I'm accepting PEP 557, Data Classes.
> 
> Eric, congrats with your efforts in proposing and implementing this PEP and 
> guiding it through the discussion! It's been great to see this idea come to 
> fruition. Thanks also to the many people who reviewed drafts or 
> implementation code, including the very generous authors and maintainers of 
> "attrs", from which this has taken many ideas.
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 557 Data Classes 5th posting

[Eric V. Smith]

On 12/4/2017 5:19 PM, Ivan Levkivskyi wrote:
Congratulations, Eric! This is a great PEP and I am looking forward to 
implement support for it in mypy ;-)


Thanks for all of your help, Ivan, especially for design decisions that 
help interoperability with mypy. I'm looking forward to mypy support, too!


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issues with PEP 526 Variable Notation at the class level

[Eric V. Smith]

On 12/7/17 3:27 PM, Raymond Hettinger wrote:
...


I'm looking for guidance or workarounds for two issues that have arisen.

First, the use of default values seems to completely preclude the use of 
__slots__.  For example, this raises a ValueError:

class A:
__slots__ = ['x', 'y']
x: int = 10
y: int = 20


Hmm, I wasn't aware of that. I'm not sure I understand why that's an 
error. Maybe it could be fixed?


Otherwise, I have a decorator that takes a dataclass and returns a new 
class with slots set:


>>> from dataclasses import dataclass
>>> from dataclass_tools import add_slots
>>> @add_slots
... @dataclass
... class C:
...   x: int = 0
...   y: int = 0
...
>>> c = C()
>>> c
C(x=0, y=0)
>>> c.z = 3
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'C' object has no attribute 'z'

This doesn't help the general case (your class A), but it does at least 
solve it for dataclasses. Whether it should be actually included, and 
what the interface would look like, can be (and I'm sure will be!) argued.


The reason I didn't include it (as @dataclass(slots=True)) is because it 
has to return a new class, and the rest of the dataclass features just 
modifies the given class in place. I wanted to maintain that conceptual 
simplicity. But this might be a reason to abandon that. For what it's 
worth, attrs does have an @attr.s(slots=True) that returns a new class 
with __slots__ set.



The second issue is that the different annotations give different signatures 
than would produced for manually written classes.  It is unclear what the best 
practice is for where to put the annotations and their associated docstrings.


I don't have any suggestions here.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issues with PEP 526 Variable Notation at the class level

[Eric V. Smith]

On 12/8/2017 1:28 PM, Raymond Hettinger wrote:




On Dec 7, 2017, at 12:47 PM, Eric V. Smith  wrote:

On 12/7/17 3:27 PM, Raymond Hettinger wrote:
...


I'm looking for guidance or workarounds for two issues that have arisen.

First, the use of default values seems to completely preclude the use of 
__slots__.  For example, this raises a ValueError:

class A:
__slots__ = ['x', 'y']
x: int = 10
y: int = 20


Hmm, I wasn't aware of that. I'm not sure I understand why that's an error. 
Maybe it could be fixed?


The way __slots__ works is that the type() metaclass automatically assigns 
member-objects to the class variables 'x' and 'y'.  Member objects are 
descriptors that do the actual lookup.

So, I don't think the language limitation can be "fixed".  Essentially, we're 
wanting to use the class variables 'x' and 'y' to hold both member objects and a default 
value.


Thanks. I figured this out after doing some research. Here's a thread 
"__slots__ and default values" from 14+ years ago from some guy named 
Hettinger:

https://mail.python.org/pipermail/python-dev/2003-May/035575.html

As to whether we add slots=True to @dataclasses, I'll let Guido decide.

The code already exists as a separate decorator here: 
https://github.com/ericvsmith/dataclasses/blob/master/dataclass_tools.py#L3, 
if you want to play with it.


Usage:

>>> @add_slots
... @dataclass
... class A:
... x: int = 10
... y: int = 20
...
>>> a = A()
>>> a
A(x=10, y=20)
>>> a.x = 15
>>> a
A(x=15, y=20)
>>> a.z = 30
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'A' object has no attribute 'z'

Folding it in to @dataclass is easy enough. On the other hand, since it 
just uses the dataclasses public API, it's not strictly required to be 
in @dataclass.



The second issue is that the different annotations give different signatures 
than would produced for manually written classes.  It is unclear what the best 
practice is for where to put the annotations and their associated docstrings.


I don't have any suggestions here.


I'm hoping the typing experts will chime in here.  The question is 
straight-forward.  Where should we look for the signature and docstring for 
constructing instances?  Should they be attached to the class, to __init__(), 
or to __new__() when it used.

It would be nice to have an official position on that before, it gets set in 
stone through arbitrary choices made by pycharm, pydoc, mypy, 
typing.NamedTuple, and dataclasses.dataclass.


I'm not sure I see why this would relate specifically to typing, since I 
don't think they'd inspect docstrings. But yes, it would be good to come 
to an agreement.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Issues with PEP 526 Variable Notation at the class level

[Eric V. Smith]

On 12/8/2017 9:14 PM, Nathaniel Smith wrote:
On Dec 7, 2017 12:49, "Eric V. Smith" <mailto:e...@trueblade.com>> wrote:


The reason I didn't include it (as @dataclass(slots=True)) is
because it has to return a new class, and the rest of the dataclass
features just modifies the given class in place. I wanted to
maintain that conceptual simplicity. But this might be a reason to
abandon that. For what it's worth, attrs does have an
@attr.s(slots=True) that returns a new class with __slots__ set.


They actually switched to always returning a new class, regardless of 
whether slots is set:


https://github.com/python-attrs/attrs/pull/260


In the end, it looks like that PR ended up just refactoring things, and 
the decision to always return a new class was deferred. I still haven't 
finished evaluating exactly what the refactoring does, though.


Eric.

You'd have to ask Hynek to get the full rationale, but I believe it was 
both for consistency with slot classes, and for consistency with regular 
class definition. For example, type.__new__ actually does different 
things depending on whether it sees an __eq__ method, so adding a method 
after the fact led to weird bugs with hashing. That class of bug goes 
away if you always set up the autogenerated methods and then call 
type.__new__.


They have a bunch of test cases that I'll have to review, too.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/10/2017 4:29 PM, Ivan Levkivskyi wrote:
On 10 December 2017 at 22:24, Raymond Hettinger 
mailto:raymond.hettin...@gmail.com>> wrote:


Without typing (only the first currently works):

     Point = namedtuple('Point', ['x', 'y', 'z'])          #
underlying store is a tuple
     Point = make_dataclass('Point', ['x', 'y', 'z'])      #
underlying store is an instance dict


Hm, I think this is a bug in implementation. The second form should also 
work.


Agreed.

I have a bunch of pending changes for dataclasses. I'll add this.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/10/2017 5:00 PM, Raymond Hettinger wrote:




On Dec 10, 2017, at 1:37 PM, Eric V. Smith  wrote:

On 12/10/2017 4:29 PM, Ivan Levkivskyi wrote:

On 10 December 2017 at 22:24, Raymond Hettinger 
mailto:raymond.hettin...@gmail.com>> wrote:
Without typing (only the first currently works):
 Point = namedtuple('Point', ['x', 'y', 'z'])  #
underlying store is a tuple
 Point = make_dataclass('Point', ['x', 'y', 'z'])  #
underlying store is an instance dict
Hm, I think this is a bug in implementation. The second form should also work.


Agreed.

I have a bunch of pending changes for dataclasses. I'll add this.

Eric.


Thanks Eric and Ivan.  You're both very responsive.  I appreciate the enormous 
efforts you're putting in to getting this right.


Thank you for your feedback. It's very helpful.

I see a couple of options:
1a: Use a default type annotation, if one is not is supplied. typing.Any 
would presumably make the most sense.

1b: Use None if not type is supplied.
2: Rework the code to not require annotations at all.

I think I'd prefer 1a, since it's easy. However, typing is not currently 
imported by dataclasses.py. There's an argument that it really needs to 
be, and I should just bite the bullet and live with it. Possibly with 
Ivan's PEP 560 work my concern on importing typing goes away.


1b would be easy, but I don't like using non-types for annotations. 2 
would be okay, but then that would be the only time __annotations__ 
wouldn't be set on a dataclass.



I suggest two other fix-ups:

1) Let make_dataclass() pass through keyword arguments to _process_class(), so 
that this will work:

 Point = make_dataclass('Point', ['x', 'y', 'z'], order=True)


Agreed.


2) Change the default value for "hash" from "None" to "False".  This might take 
a little effort because there is currently an oddity where setting hash=False causes it to be hashable.  I'm 
pretty sure this wasn't intended ;-)


It's sufficiently confusing that I need to sit down when I have some 
free time and noodle this through. But it's still on my radar.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/10/2017 5:00 PM, Raymond Hettinger wrote:




On Dec 10, 2017, at 1:37 PM, Eric V. Smith  wrote:

On 12/10/2017 4:29 PM, Ivan Levkivskyi wrote:

On 10 December 2017 at 22:24, Raymond Hettinger 
mailto:raymond.hettin...@gmail.com>> wrote:
Without typing (only the first currently works):
 Point = namedtuple('Point', ['x', 'y', 'z'])  #
underlying store is a tuple
 Point = make_dataclass('Point', ['x', 'y', 'z'])  #
underlying store is an instance dict
Hm, I think this is a bug in implementation. The second form should also work.


Agreed.

I have a bunch of pending changes for dataclasses. I'll add this.


This is bpo-32278.


I suggest two other fix-ups:

1) Let make_dataclass() pass through keyword arguments to _process_class(), so 
that this will work:

 Point = make_dataclass('Point', ['x', 'y', 'z'], order=True)


This is bpo-32279.


2) Change the default value for "hash" from "None" to "False".  This might take 
a little effort because there is currently an oddity where setting hash=False causes it to be hashable.  I'm 
pretty sure this wasn't intended ;-)


No time for this one yet. Soon!

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/11/2017 9:25 PM, Nick Coghlan wrote:
On 11 Dec. 2017 12:26 pm, "Eric V. Smith" <mailto:e...@trueblade.com>> wrote:




I see a couple of options:
1a: Use a default type annotation, if one is not is supplied.
typing.Any would presumably make the most sense.
1b: Use None if not type is supplied.
2: Rework the code to not require annotations at all.


1c: annotate with the string "typing.Any" (this may require a tweak to 
the rules for evaluating lazy annotations, though)


Good idea, since it needs to be supported, anyway, especially in light 
of PEP 563.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/15/2017 5:56 AM, Steve Holden wrote:
On Mon, Dec 11, 2017 at 5:10 PM, Chris Barker - NOAA Federal 
mailto:chris.bar...@noaa.gov>> wrote:

...

However, typing is not currently imported by dataclasses.py.

>

And there you have an actual reason besides my uneasiness :-)

- CHB

​hmm...​


[Agreed with Antoine on the MUA and quoting being confusing.]

The only reason typing isn't imported is performance. I hope that once 
PEP 560 is complete this will no longer be an issue, and dataclasses 
will always import typing. But of course typing will still not be needed 
for most uses of @dataclass or make_dataclass(). This is explained in 
the PEP.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Guarantee ordered dict literals in v3.7?

[Eric V. Smith]

On 12/15/2017 11:55 AM, Guido van Rossum wrote:
On Fri, Dec 15, 2017 at 8:32 AM, Raymond Hettinger 
mailto:raymond.hettin...@gmail.com>> wrote:



> On Dec 15, 2017, at 7:53 AM, Guido van Rossum mailto:gu...@python.org>> wrote:
>
> Make it so. "Dict keeps insertion order" is the ruling. Thanks!

Thank you.  That is wonderful news :-)

Would it be reasonable to replace some of the OrderedDict() uses in
the standard library with dict()?  For example, have namedtuples's
_asdict() go back to returning a plain dict as it did in its
original incarnation. Also, it looks like argparse could save an
import by using a regular dict.


If it's documented as OrderedDict that would be backwards incompatible, 
since that has additional methods. Even if not documented it's likely to 
break some code. So, I'm not sure about this (though I agree with the 
sentiment that OrderedDict is much less important now).


For dataclasses, I'll change from OrderedDict to dict, since there's no 
backward compatibility concern.


But I need to remember to not do that when I put the 3.6 version on PyPI.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/18/2017 2:55 PM, Ivan Levkivskyi wrote:
On 18 December 2017 at 20:38, Nick Coghlan > wrote:



On 19 Dec. 2017 7:00 am, "Chris Barker" mailto:chris.bar...@noaa.gov>> wrote:


Are there other options??

plain old:

@dataclass
class C:
     a = 1
     b = 1.0

would work, though then there would be no way to express fields
without defaults:


The PEP already supports using "a = field(); b = field()" (etc) to
declare untyped fields without a default value.


The PEP is not 100% clear not this, but it is currently not the case and 
this may be intentional (one obvious way to do it),

I just tried and this does not work:

@dataclass
class C:
     x = field()

generates `__init__` etc. with no arguments. I think however that it is 
better to generate an error than silently ignore it.

(Or if this a bug in the implementation, it should be just fixed.)


Hmm, not sure why that doesn't generate an error. I think it's a bug 
that should be fixed. Or, we could make the same change we're making in 
make_dataclass(), where we'll use "typing.Any" (as a string) if the type 
is omitted. See https://bugs.python.org/issue32278.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/18/2017 9:41 PM, Chris Barker wrote:

I'm really surprised no one seems to get my point here.

TL;DR:
My point is that having type annotation syntax required for something in 
the stdlib is a significant step toward "normalizing" type hinting in 
Python. Whether that's a good idea or not is a judgement call, but it IS 
a big step.


I get your point, I'm just not concerned about it.

I also don't think it's surprising that you can put misleading 
information (including non-types) in type annotations. All of the 
documentation and discussions are quite clear that type information is 
ignored at runtime.


It _is_ true that @dataclass does actually inspect the type at runtime, 
but those uses are very rare. And if you do need them, the actual type T 
used by ClassVar[T] and InitVar[T] are still ignored.


Data Classes is also not the first use of type annotations in the stdlib:
https://docs.python.org/3/library/typing.html#typing.NamedTuple

When I say that "typing is optional", I mean importing the typing 
module, not that annotations are optional.


Eric.


@Chris

People are still allowed not to use dataclasses if they really don't
like type hints :-)
Seriously however, annotations are just syntax. In this sense PEP
526 is more like PEP 3107,
and less like PEP 484. People are still free to write:

@dataclass
class C:
     x: "first coordinate"
     y: "second coordinate"
     plus: "I don't like types"


Well, yes, of course, but this is not like PEP 3107, as it introduces a 
requirement for annotations (maybe not *type* annotations per se) in the 
std lib. Again, that may be the best way to go -- but it should be done 
deliberately.


@dataclass

class C:
     x: ...
     y: ...


Ah! I had no idea you could use ellipses to indicate no type. That 
actually helps a lot. We really should have that prominent in the docs. 
And in the dataclass docs, not just the type hinting docs -- again, 
people will want to use these that may not have any interest in nor 
prior knowledge of type hints.


I don't see so big difference between hypothesis (testing lib) using
annotations for their purposes
from the situation with dataclasses.


The big difference is that hypothesis is not in the standard library. 
Also, I didn't know about hypothesis until just now, but their very 
first example in the quick start does not use annotation syntax, so it's 
not as baked in as it is with dataclasses.


If you have ideas about how to improve the dataclass docs, this can
be discussed in the issue https://bugs.python.org/issue32216



I'll try to find time to contribute there -- though maybe better to have 
the doc draft in gitHub?


 > ... the type will in fact be completely ignored by the
implementation.
> Newbies are going to be confused by this -- they really are.

This is not different from

def f(x: int):
     pass

f("What")  # OK

that exists starting from Python 3.0. Although I agree this is
confusing, the way forward could be just explaining this better in
the docs.


Again the difference is that EVERY introduction to defining python 
functions doesn't use the type hint. And even more to the point, you CAN 
define a function without any annotations.


But frankly, I think as type hinting becomes more common, we're going to 
see a lot of confusion :-(


If you want my personal opinion about the current situation about
type hints _in general_, then I can say that
I have seen many cases where people use type hints where they are
not needed
(for example in 10 line scripts or in highly polymorphic functions),
so I agree that some community
style guidance (like PEP 8) may be helpful. 



It's going to get worse before it gets better :-(

@dataclass
class C:
  x = field()


that does require that `field` be imported, so not as nice. I kinda like 
the ellipses better.


but good to have a way.

-Chris


--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

chris.bar...@noaa.gov 


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Guarantee ordered dict literals in v3.7?

[Eric V. Smith]

On 12/19/2017 2:38 AM, Steve Dower wrote:

On 18Dec2017 2309, Steven D'Aprano wrote:

[A LOT OF THINGS I AGREE WITH]

I agree completely with Steven's reasoning here, and it bothers me that
what is an irrelevant change to many users (dict becoming ordered) seems
to imply that all users of dict have to be updated.

I have never needed OrderedDict before, and dict now also being ordered
doesn't mean that when I reach for it I'm doing it because I need an
ordered dict - I probably just need a regular dict. *Nothing* about dict
should change for me between versions.

Adding an option to pprint to explicitly control sorting without
changing the default is fine. Please stop assuming that everyone wants
an OrderedDict when they say dict. It's an invalid assumption.


Well said, Steve and Steven. I completely agree.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/20/2017 6:57 PM, Mike Miller wrote:

On 2017-12-19 02:53, Paul Moore wrote:

Also, the fact that no-one raised this issue during the whole time the
PEP was being discussed (at least as far as I recollect) and that
Guido (who of all of us should be most aware of what is and isn't
acceptable use of annotations in the stdlib) approved the PEP,
suggests to me that this isn't that big a deal.



Hi, I asked about this in the first posting of the PEP and agree with 
Chris.


 
https://mail.python.org/pipermail/python-dev/2017-September/149406.html



There is definitely a passive bias towards using types with dataclasses 
in that the Eric (the author) doesn't appear to want an example without 
them in the pep/docs.


I'm not sure what such an example would look like. Do you mean without 
annotations? Or do you mean without specifying the "correct" type, like:


@dataclass
class C:
   x: int = 'hello world'

?

Or something else?

Can you provide an example of what you'd like to see?

It seems that typing proponents are sufficiently enamored with them that 
they can't imagine anyone else feeling differently, haha.


I've never used typing or mypy, so you're not talking about me. I do 
like the conciseness that annotations bring to dataclasses, though. If 
you buy that (and you might not), then I don't see the point of not 
using a correct type annotation.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/20/2017 8:13 PM, Eric V. Smith wrote:

There is definitely a passive bias towards using types with 
dataclasses in that the Eric (the author) doesn't appear to want an 
example without them in the pep/docs.


I'm not sure what such an example would look like. Do you mean without 
annotations? Or do you mean without specifying the "correct" type, like:


@dataclass
class C:
    x: int = 'hello world'

?

Or something else?

Can you provide an example of what you'd like to see?


Re-reading my post you referenced, is it just an example using 
typing.Any? I'm okay with that in the docs, I just didn't want to focus 
on it in the PEP. I want the PEP to only have the one reference to 
typing, for typing.ClassVar. I figure the people reading the PEP can 
extrapolate to all of the possible uses for annotations that they don't 
need to see a typing.Any example.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/21/2017 1:46 AM, Chris Barker wrote:
On Wed, Dec 20, 2017 at 5:29 PM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


There is definitely a passive bias towards using types with
dataclasses in that the Eric (the author) doesn't appear to want an
example without them in the pep/docs.


I'm not sure what such an example would look like. Do you mean
without annotations? 



IIUC, there is not way to make a dataclass without annotations, yes? 
That it, using annotations to determine the fields is the one and only 
way the decorator works. So it's impossible to give an example without 
annotations, yes?


Correct. Well, you will be able to use make_dataclass() without type 
information after I fix bpo-32278, but most users won't be using that.


I suggest that it be clear in the docs, and ideally in the PEP, that the 
dataclass decorator is using the *annotation" syntax, and that the the 
only relevant part it uses is that an annotation exists, but the value 
of the annotation is essentially (completely?) ignored. 


I think the PEP is very clear about this: "The dataclass decorator 
examines the class to find fields. A field is defined as any variable 
identified in __annotations__. That is, a variable that has a type 
annotation. With two exceptions described below, none of the Data Class 
machinery examines the type specified in the annotation."


I agree the docs should also be clear about this.

So we should 
have examples like:


@dataclass
class C:
     a: ...  # field with no default
     b: ... = 0 # filed with a default value

Then maybe:

@dataclass
class C:
     a: "the a parameter" # field with no default
     b: "another, different parameter" = 0.0 # field with a default

Then the docs can go to say that if the user wants to specify a type for 
use with a static type checking pre-processor, they can do it like so:


@dataclass
class C:
     a: int # integer field with no default
     b: float = 0.0 # float field with a default

And the types will be recognized by type checkers such as mypy.

And I think the non-typed examples should go first in the docs.


I'll leave this for others to decide. The docs, and how approachable 
they are to various audiences, isn't my area of expertise.


This is completely analogous to how all the other parts of python are 
taught. Would anyone suggest that the very first example of a function 
definition that a newbie sees would be:


def func(a: int, b:float = 0.0):
     body_of_function

Then, _maybe_ way down on the page, you mention that oh, by the way, 
those types are completely ignored by Python. And not even give any 
examples without types?



 > Re-reading my post you referenced, is it just an example using 
typing.Any?


I actually think that is exactly the wrong point -- typing.Any is still 
using type hinting -- it's an explicit way to say, "any type will do", 
but it's only relevant if you are using a type checker. We really need 
examples for folks that don't know or care about type hinting at all.


typing.Any is for use by people that are explicitly adding type hinting, 
and should be discussed in type hinting documentation.


 >  I'm okay with that in the docs, I just didn't want to focus on it in 
the PEP. I want the PEP to only
 > have the one reference to typing, for typing.ClassVar. I figure the 
people reading the PEP can
 > extrapolate to all of the possible uses for annotations that they 
don't need to see a typing.Any

 > example.

no they don't, but they DO need to see examples without type hints at all.


I'm not opposed to this in the documentation. Maybe we should decide on 
a convention on what to use to convey "don't care". I've seen 
typing.Any, None, ellipsis, strings, etc. all used.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/21/17 6:25 AM, Sven R. Kunze wrote:

On 21.12.2017 11:22, Terry Reedy wrote:



@dataclass
class C:
 a: int # integer field with no default
 b: float = 0.0 # float field with a default

And the types will be recognized by type checkers such as mypy.

And I think the non-typed examples should go first in the docs.




I still don't understand why "I don't care" can be defined by "leaving out"

@dataclass
class C:
 b = 0.0 # float field with a default


Because you can't know the order that x and y are defined in this example:

class C:
x: int
y = 0

'x' is not in C.__dict__, and 'y' is not in C.__annotations__.

Someone will suggest a metaclass, but that has its own problems. Mainly, 
interfering with other metaclasses.


Eric.





For non-default fields, I like ellipsis too.

Cheer,
Sven


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] is typing optional in dataclasses?

[Eric V. Smith]

On 12/21/2017 7:55 PM, MRAB wrote:

On 2017-12-22 00:19, Gregory P. Smith wrote:

(subject for this sub-thread updated)

On Thu, Dec 21, 2017 at 4:08 PM Chris Barker > wrote:


    On Thu, Dec 21, 2017 at 3:36 PM, Gregory P. Smith mailto:g...@krypto.org>> wrote:

     But we already have ... which does - so I'd suggest that for
    people who are averse to importing anything from typing and
    using the also quite readable Any.  (ie: document this as the
    expected practice with both having the same meaning)


    I don't think they do, actually - I haven't been following the
    typing discussions, but someone in this thread said that ... means
    "use the type of teh default" or something like that.


indeed, they may not.  though if that is the definition is it 
reasonable to say that type analyzers recognize the potential 
recursive meaning when the _default_ is ... and treat that as Any?


another option that crossed my mind was "a: 10" without using =.  but 
that really abuses __attributes__ by sticking the default value in 
there which the @dataclass decorator would presumably immediately need 
to undo and fix up before returning the class.  but I don't find 
assigning a value without an = sign to be pythonic so please lets not 
do that! :)


If you allowed "a: 10" (an int value), then you might also allow "a: 
'foo'" (a string value), but wouldn't that be interpreted as a type 
called "foo"?


As far as dataclasses are concerned, both of these are allowed, and 
since neither is ClassVar or InitvVar, they're ignored. Type checkers 
would object to the int, and I assume also the string unless there was a 
type foo defined. See 
https://www.python.org/dev/peps/pep-0484/#the-problem-of-forward-declarations 
and typing.get_type_hints().


It's a bug that dataclasses currently does not inspect string 
annotations to see if they're actually ClassVar or InitVar declarations. 
PEP 563 makes it critical (and not merely important) to look at the 
string annotations. Whether or not that involves typing.get_type_hints() 
or not, I haven't yet decided. I'm waiting for PEPs 563 and 560 to be 
implemented before taking another look at it.


Eric.



If you can't have a string value, then you shouldn't have an int value 
either.


[snip]

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com 



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/21/2017 6:36 AM, Ivan Levkivskyi wrote:
On 21 December 2017 at 11:22, Terry Reedy <mailto:tjre...@udel.edu>> wrote:


On 12/21/2017 4:22 AM, Eric V. Smith wrote:

On 12/21/2017 1:46 AM, Chris Barker wrote:


I suggest that it be clear in the docs, and ideally in the
PEP, that the dataclass decorator is using the *annotation"
syntax, and that the the only relevant part it uses is that
an annotation exists, but the value of the annotation is
essentially (completely?) ignored.


I think the PEP is very clear about this: "The dataclass
decorator examines the class to find fields. A field is defined
as any variable identified in __annotations__. That is, a
variable that has a type annotation. With two exceptions
described below, none of the Data Class machinery examines the
type specified in the annotation."


This seems clear enough.  It could come after describing what a
dataclass *is*.

I agree the docs should also be clear about this.



So we should have examples like:

@dataclass
class C:
  a: ...  # field with no default
  b: ... = 0 # filed with a default value

Then maybe:

@dataclass
class C:
  a: "the a parameter" # field with no default
  b: "another, different parameter" = 0.0 # field with a
default

Then the docs can go to say that if the user wants to
specify a type for use with a static type checking
pre-processor, they can do it like so:

@dataclass
class C:
  a: int # integer field with no default
  b: float = 0.0 # float field with a default

And the types will be recognized by type checkers such as mypy.

And I think the non-typed examples should go first in the docs.


Module some bike-shedding, the above seems pretty good to me.


For me, the three options for "don't care" have a bit different meaning:

* typing.Any: this class is supposed to be used with static type 
checkers, but this field is too dynamic
* ... (ellipsis): this class may or may not be used with static type 
checkers, use the inferred type in the latter case

* "field docstring": this class should not be used with static type checkers

Assuming this, the second option would be the "real" "don't care". If 
this makes sense,
then we can go the way proposed in 
https://github.com/python/typing/issues/276 and make ellipsis semantics 
"official" in PEP 484.

(pending Guido's approval)


In 
https://github.com/ericvsmith/dataclasses/issues/2#issuecomment-353918024, 
Guido has suggested using `object`, which has the benefit of not needing 
an import. And to me, it communicates the "don't care" aspect well 
enough. I do understand the difference if you're using a type checker 
(see for example 
https://stackoverflow.com/questions/39817081/typing-any-vs-object), but 
if you care about that, use typing.Any.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

I’m traveling until next week, and haven’t had time to read any of these 
emails. I’ll look at them when I return. 

--
Eric.

> On Dec 30, 2017, at 5:20 AM, Raymond Hettinger  
> wrote:
> 
> 
>> On Dec 29, 2017, at 4:52 PM, Guido van Rossum  wrote:
>> 
>> I still think it should overrides anything that's just inherited but nothing 
>> that's defined in the class being decorated.
> 
> This has the virtue of being easy to explain, and it will help with debugging 
> by honoring the code proximate to the decorator :-)
> 
> For what it is worth, the functools.total_ordering class decorator does 
> something similar -- though not exactly the same.  A root comparison method 
> is considered user-specified if it is different than the default method 
> provided by object: 
> 
>def total_ordering(cls):
>"""Class decorator that fills in missing ordering methods"""
># Find user-defined comparisons (not those inherited from object).
>roots = {op for op in _convert if getattr(cls, op, None) is not 
> getattr(object, op, None)}
>...
> 
> The @dataclass decorator has a much broader mandate and we have almost no 
> experience with it, so it is hard to know what legitimate use cases will 
> arise.
> 
> 
> Raymond
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

I’ll open an issue after I have time to read this thread and comment on it. 

--
Eric.

> On Jan 3, 2018, at 12:31 PM, Ivan Levkivskyi  wrote:
> 
> I like the Guido's proposal, i.e.
> 
> if '__repr__' not in cls.__dict__:
> ...  # generate the method
> 
> etc. I didn't find an issue to track this. Maybe we should open one?
> 
> --
> Ivan
> 
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/2/2018 12:01 AM, Guido van Rossum wrote:
On Mon, Jan 1, 2018 at 8:50 PM, Ethan Smith > wrote:




On Mon, Jan 1, 2018 at 5:03 PM, Chris Barker mailto:chris.bar...@noaa.gov>> wrote:

On Sat, Dec 30, 2017 at 7:27 AM, Stephen J. Turnbull
mailto:turnbull.stephen...@u.tsukuba.ac.jp>> wrote:

   Just use the simple rule that a new
__repr__ is generated unless provided in the dataclass.


are we only talking about __repr__ here ???

I interpreted Guido's proposal as being about all methods -- we
_may_ want something special for __repr__, but I hope not.

[...]


I interpreted this to be for all methods as well, which makes sense.
Special casing just __repr__ doesn't make sense to me, but I will
wait for Guido to clarify.


Indeed, I just wrote __repr__ for simplicity. This should apply to all 
special methods. (Though there may be some complications for 
__eq__/__ne__ and for the ordering operators.)


On Mon, Jan 1, 2018 at 9:44 PM, Chris Barker > wrote:


On Mon, Jan 1, 2018 at 7:50 PM, Ethan Smith mailto:et...@ethanhs.me>> wrote:


Will you get the "right" __repr__ now if you derive a
dataclass from a dataclass? That would be a nice feature.


The __repr__ will be generated by the child dataclass unless the
user overrides it. So I believe this is the "right" __repr__.


what I was wondering is if the child will know about all the fields
in the parent -- so it could make a full __repr__.


Yes, there's a class variable (__dataclass_fields__) that identifies the 
parent fields. The PEP doesn't mention this or the fact that special 
methods (like __repr__ and __init__) can tell whether a base class is a 
dataclass. It probably should though. (@Eric)


I think that's covered in this section: 
https://www.python.org/dev/peps/pep-0557/#inheritance


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 11:24 AM, Guido van Rossum wrote:
On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


On 1/2/2018 12:01 AM, Guido van Rossum wrote:

Yes, there's a class variable (__dataclass_fields__) that
identifies the parent fields. The PEP doesn't mention this or
the fact that special methods (like __repr__ and __init__) can
tell whether a base class is a dataclass. It probably should
though. (@Eric)


I think that's covered in this section:
https://www.python.org/dev/peps/pep-0557/#inheritance
<https://www.python.org/dev/peps/pep-0557/#inheritance>


I was specifically talking about the name and contents of 
__dataclass_fields__, which are not documented by the PEP. I expect it's 
inevitable that people will be looking at this (since they can see it in 
the source code). Or do you recommend that people use 
dataclasses.fields() and catch ValueError?


The expectation is to use dataclasses.fields(). Both it and 
__dataclass_fields__ contain the fields for this class and the parents. 
The only difference is the pseudo-fields.


I can add some words describing .fields() returning which fields are 
present.


I notice that _isdataclass() 
exists but is private and I don't recall why. 


I think the argument was that it's an anti-pattern, and if you really 
want to know, just call dataclasses.fields() and catch the TypeError. I 
have this in a helper file:


def isdataclass(obj):
"""Returns True for dataclass classes and instances."""
try:
dataclasses.fields(obj)
return True
except TypeError:
return False


(Also now I'm curious what

the "pseudo-fields" are that fields() ignores, but that's OT.)


ClassVar and InitVar "fields". dataclasses.fields() doesn't return them.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 12:58 PM, Guido van Rossum wrote:
Hm. I don't know that people will conclude that checking for a dataclass 
is an anti-pattern. They'll probably just invent a myriad of different 
hacks like the one you showed. I recommend making it public.


I'm trying to track down the original discussion. We got bogged down on 
whether it worked for classes or instances or both, then we got tied up 
in naming it (surprise!), then it looks like we decided to just not 
include it since you could make those decisions for yourself.


I think the discussion is buried in this thread:
https://mail.python.org/pipermail/python-dev/2017-November/150966.html

Which references:
https://github.com/ericvsmith/dataclasses/issues/99

So, ignoring the naming issue, I think if we want to revive it, the 
question is: should isdataclass() return True on just instances, just 
classes, or both? And should it ever raise an exception, or just return 
False?


I still worry a bit about ClassVar and InitVar being potentially useful 
but I concede I have no use case so I'll drop it.


IIRC, we decided that we could add a parameter to dataclasses.fields() 
if we ever wanted to return pseudo-fields. But no one came up with a use 
case.


Eric.



On Fri, Jan 5, 2018 at 8:43 AM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


On 1/5/2018 11:24 AM, Guido van Rossum wrote:

On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith
mailto:e...@trueblade.com>
<mailto:e...@trueblade.com <mailto:e...@trueblade.com>>> wrote:

     On 1/2/2018 12:01 AM, Guido van Rossum wrote:

         Yes, there's a class variable (__dataclass_fields__) that
         identifies the parent fields. The PEP doesn't mention
this or
         the fact that special methods (like __repr__ and
__init__) can
         tell whether a base class is a dataclass. It probably
should
         though. (@Eric)


     I think that's covered in this section:
https://www.python.org/dev/peps/pep-0557/#inheritance
<https://www.python.org/dev/peps/pep-0557/#inheritance>
     <https://www.python.org/dev/peps/pep-0557/#inheritance
<https://www.python.org/dev/peps/pep-0557/#inheritance>>


I was specifically talking about the name and contents of
__dataclass_fields__, which are not documented by the PEP. I
expect it's inevitable that people will be looking at this
(since they can see it in the source code). Or do you recommend
that people use dataclasses.fields() and catch ValueError?


The expectation is to use dataclasses.fields(). Both it and
__dataclass_fields__ contain the fields for this class and the
parents. The only difference is the pseudo-fields.

I can add some words describing .fields() returning which fields are
present.

I notice that _isdataclass() exists but is private and I don't
recall why.


I think the argument was that it's an anti-pattern, and if you
really want to know, just call dataclasses.fields() and catch the
TypeError. I have this in a helper file:

def isdataclass(obj):
     """Returns True for dataclass classes and instances."""
     try:
         dataclasses.fields(obj)
         return True
     except TypeError:
         return False


(Also now I'm curious what

the "pseudo-fields" are that fields() ignores, but that's OT.)


ClassVar and InitVar "fields". dataclasses.fields() doesn't return them.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org <mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
<https://mail.python.org/mailman/listinfo/python-dev>
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org 
<https://mail.python.org/mailman/options/python-dev/guido%40python.org>




--
--Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/eric%2Ba-python-dev%40trueblade.com



___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 2:09 PM, Guido van Rossum wrote:
I'm normally no big fan of things that take either a class or an 
instance, but since fields() does this, I think is_dataclass() should 
to. And that's the name I'd choose. OK on the pseudo-fields.


Sounds good. I'll open a bpo issue.

Eric.

On Fri, Jan 5, 2018 at 11:06 AM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


On 1/5/2018 12:58 PM, Guido van Rossum wrote:

Hm. I don't know that people will conclude that checking for a
dataclass is an anti-pattern. They'll probably just invent a
myriad of different hacks like the one you showed. I recommend
making it public.


I'm trying to track down the original discussion. We got bogged down
on whether it worked for classes or instances or both, then we got
tied up in naming it (surprise!), then it looks like we decided to
just not include it since you could make those decisions for yourself.

I think the discussion is buried in this thread:
https://mail.python.org/pipermail/python-dev/2017-November/150966.html
<https://mail.python.org/pipermail/python-dev/2017-November/150966.html>

Which references:
https://github.com/ericvsmith/dataclasses/issues/99
<https://github.com/ericvsmith/dataclasses/issues/99>

So, ignoring the naming issue, I think if we want to revive it, the
question is: should isdataclass() return True on just instances,
just classes, or both? And should it ever raise an exception, or
just return False?

I still worry a bit about ClassVar and InitVar being potentially
useful but I concede I have no use case so I'll drop it.


IIRC, we decided that we could add a parameter to
dataclasses.fields() if we ever wanted to return pseudo-fields. But
no one came up with a use case.

Eric.


On Fri, Jan 5, 2018 at 8:43 AM, Eric V. Smith
mailto:e...@trueblade.com>
<mailto:e...@trueblade.com <mailto:e...@trueblade.com>>> wrote:

     On 1/5/2018 11:24 AM, Guido van Rossum wrote:

         On Fri, Jan 5, 2018 at 5:08 AM, Eric V. Smith
         mailto:e...@trueblade.com>
<mailto:e...@trueblade.com <mailto:e...@trueblade.com>>
         <mailto:e...@trueblade.com <mailto:e...@trueblade.com>
<mailto:e...@trueblade.com <mailto:e...@trueblade.com>>>> wrote:

              On 1/2/2018 12:01 AM, Guido van Rossum wrote:

                  Yes, there's a class variable
(__dataclass_fields__) that
                  identifies the parent fields. The PEP doesn't
mention
         this or
                  the fact that special methods (like __repr__ and
         __init__) can
                  tell whether a base class is a dataclass. It
probably
         should
                  though. (@Eric)


              I think that's covered in this section:
https://www.python.org/dev/peps/pep-0557/#inheritance
<https://www.python.org/dev/peps/pep-0557/#inheritance>
         <https://www.python.org/dev/peps/pep-0557/#inheritance
<https://www.python.org/dev/peps/pep-0557/#inheritance>>
 
<https://www.python.org/dev/peps/pep-0557/#inheritance

<https://www.python.org/dev/peps/pep-0557/#inheritance>
         <https://www.python.org/dev/peps/pep-0557/#inheritance
<https://www.python.org/dev/peps/pep-0557/#inheritance>>>


         I was specifically talking about the name and contents of
         __dataclass_fields__, which are not documented by the
PEP. I
         expect it's inevitable that people will be looking at this
         (since they can see it in the source code). Or do you
recommend
         that people use dataclasses.fields() and catch ValueError?


     The expectation is to use dataclasses.fields(). Both it and
     __dataclass_fields__ contain the fields for this class and the
     parents. The only difference is the pseudo-fields.

     I can add some words describing .fields() returning which
fields are
     present.

         I notice that _isdataclass() exists but is private and
I don't
         recall why.


     I think the argument was that it's an anti-pattern, and if you
     really want to know, just call dataclasses.fields() and
catch the
     TypeError. I have this in a helper file:

     def isdataclass(obj):
          """Returns True for dataclass classes and instances."""
          try:
              dataclasses

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/5/2018 2:11 PM, Eric V. Smith wrote:

On 1/5/2018 2:09 PM, Guido van Rossum wrote:
I'm normally no big fan of things that take either a class or an 
instance, but since fields() does this, I think is_dataclass() should 
to. And that's the name I'd choose. OK on the pseudo-fields.


Sounds good. I'll open a bpo issue.


https://bugs.python.org/issue32499

I'm slowly reading through the rest of this thread and will respond this 
weekend.


Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Eric V. Smith]

On 12/10/2017 5:00 PM, Raymond Hettinger wrote:




On Dec 10, 2017, at 1:37 PM, Eric V. Smith  wrote:

On 12/10/2017 4:29 PM, Ivan Levkivskyi wrote:

On 10 December 2017 at 22:24, Raymond Hettinger 
mailto:raymond.hettin...@gmail.com>> wrote:
Without typing (only the first currently works):
 Point = namedtuple('Point', ['x', 'y', 'z'])  #
underlying store is a tuple
 Point = make_dataclass('Point', ['x', 'y', 'z'])  #
underlying store is an instance dict
Hm, I think this is a bug in implementation. The second form should also work.


Agreed.


I've checked this under bpo-32278.



I have a bunch of pending changes for dataclasses. I'll add this.

Eric.


Thanks Eric and Ivan.  You're both very responsive.  I appreciate the enormous 
efforts you're putting in to getting this right.

I suggest two other fix-ups:

1) Let make_dataclass() pass through keyword arguments to _process_class(), so 
that this will work:

 Point = make_dataclass('Point', ['x', 'y', 'z'], order=True)


And I've checked this in under bpo-32279.


2) Change the default value for "hash" from "None" to "False".  This might take 
a little effort because there is currently an oddity where setting hash=False causes it to be hashable.  I'm 
pretty sure this wasn't intended ;-)


I haven't looked at this yet.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/3/2018 1:17 PM, Eric V. Smith wrote:

I’ll open an issue after I have time to read this thread and comment on it.


https://bugs.python.org/issue32513
I need to think though how __eq__ and __ne__ work, as well as the 
ordering operators.


My specific concern with __ne__ is that there's one flag to control 
their generation, but python will use "not __eq__" if you don't provide 
__ne__. I need to think through what happens if the user only provides 
__eq__: does dataclasses do nothing, does it add __ne__, and how does 
this interact with a base class that does provide __ne__.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Eric V. Smith]

On 1/7/2018 12:25 PM, Guido van Rossum wrote:
On Sun, Jan 7, 2018 at 9:09 AM, Eric V. Smith <mailto:e...@trueblade.com>> wrote:


On 1/3/2018 1:17 PM, Eric V. Smith wrote:

I’ll open an issue after I have time to read this thread and
comment on it.


https://bugs.python.org/issue32513 <https://bugs.python.org/issue32513>
I need to think though how __eq__ and __ne__ work, as well as the
ordering operators.

My specific concern with __ne__ is that there's one flag to control
their generation, but python will use "not __eq__" if you don't
provide __ne__. I need to think through what happens if the user
only provides __eq__: does dataclasses do nothing, does it add
__ne__, and how does this interact with a base class that does
provide __ne__.


Maybe dataclasses should only ever provide __eq__ and always assume 
Python's default for __ne__ kicks in? If that's not acceptable (maybe 
there are cases where a user did write an explicit __ne__ that needs to 
be overridden) I would recommend the following rule:


- If there's an __eq__, don't do anything (regardless of whether there's 
an __ne__)
- If there no __eq__ but there is an __ne__, generate __eq__ but don't 
generate __ne__

- If neither exists, generate both


I've added my proposal on issue 32513:
https://bugs.python.org/issue32513#msg310392

It's long, so I won't repeat it here. The only really confusing part is 
__hash__ and its interaction with __eq__.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GH-NNNN vs #NNNN in merge commit

[Eric V. Smith]

On 1/25/2018 8:13 PM, Terry Reedy wrote:

On 1/25/2018 7:47 PM, Mariatta Wijaya wrote:
I think we're starting to deviate from the original topic here which 
is: please replace # with GH- when you click Squash & Merge button.


I will try to remember to do this, although it seems pointless if most 
people do not.


I sometimes forget, too.

Since changing this in the github workflow seems hard, how about sending 
an email to the committer after the commit if this mistake is detected? 
I know that in my case, if I were reminded a few times, I would be 
better at doing it correctly in the future.


Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] GH-NNNN vs #NNNN in merge commit

[Eric V. Smith]

On 1/26/2018 12:22 PM, Mariatta Wijaya wrote:

I believe https://github.com/python/bedevere/pull/82
 will add a comment,
which will get emailed to everyone nosy on the PR.

Yes, and I've just updated my PR description:

If a PR was merged and the commit message was not changed from 
|#| to |GH-|, bedevere-bot will leave a comment to remind the 
core dev to update it next time.

For example:
```
@Mariatta: Please replace # with GH- in the commit message next time. 
Thanks!

```

Does that work for everyone?


It works for me: I think this is very helpful. Thanks for coding it up 
so quickly!


Eric
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com