date:20190106

Hello,

As explained in
https://docs.python.org/3/reference/compound_stmts.html#the-try-statement ,

except E as N:
foo

is actually compiled as:

except E as N:
try:
foo
finally:
del N

But looking at the generated bytecode, it's worse than that:

 16 STORE_NAME   1 (N)
 18 POP_TOP
 20 SETUP_FINALLY8 (to 30)

  4  22 LOAD_NAME2 (foo)
 24 POP_TOP
 26 POP_BLOCK
 28 LOAD_CONST   0 (None)
>>   30 LOAD_CONST   0 (None)
 32 STORE_NAME   1 (N)
 34 DELETE_NAME  1 (N)
 36 END_FINALLY


It's clear that what happens there is that first None is stored to N,
just to del it as the next step. Indeed, that's what done in the
compile.c:

https://github.com/python/cpython/blob/master/Python/compile.c#L2905

Storing None looks superfluous. For example, DELETE_FAST explicitly
stores NULL on delete.
https://github.com/python/cpython/blob/master/Python/ceval.c#L2249 

Likewise, for DELETE_NAME/DELETE_GLOBAL, PyObject_DelItem() is
called which translates to:

m->mp_ass_subscript(o, key, (PyObject*)NULL);

So hopefully any compliant mapping implementation actually clears
stored value, not leaving it dangling.

The "store None" behavior can be traced down to introduction of the
"del except target var" behavior back in 2007:
https://github.com/python/cpython/commit/b940e113bf90ff71b0ef57414ea2beea9d2a4bc0#diff-cb296cc5109f5640ff3f6d7198a6abeeR1999

There's no clear explanation why it's done like that, so probably an
artifact of the initial implementation. Note that even
https://github.com/python/cpython/commit/520b7ae27e39d1c77ea74ccd1b184d7cb43f9dcb
which did quite a bunch of refactoring to "except" implementation, and
reformatted this code, otherwise left it in place.

P.S.
I actually wanted to argue that such an implementation is hardly ideal
at all. Consider:


e = "my precious data"

try:
1/0
except Exception as e:
pass

# Where's my *global* variable?
# Worse, my variable can be gone or not, depending on whether exception
# triggered or not.
print(e)


So, perhaps the change should be not removing "e = None" part, but
conversely, removing the "del e" part.


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On Sun, Jan 6, 2019 at 10:12 PM Paul Sokolovsky  wrote:
> It's clear that what happens there is that first None is stored to N,
> just to del it as the next step. Indeed, that's what done in the
> compile.c:
>
> https://github.com/python/cpython/blob/master/Python/compile.c#L2905
>
> Storing None looks superfluous.
>
> There's no clear explanation why it's done like that, so probably an
> artifact of the initial implementation.

With something as clearly deliberate as this, it's generally safe to
assume there's a good reason for it. But I would very much like to see
that reason documented, preferably with an example of something that
could otherwise fail.

> P.S.
> I actually wanted to argue that such an implementation is hardly ideal
> at all. Consider:
>
> 
> e = "my precious data"
>
> try:
> 1/0
> except Exception as e:
> pass
>
> # Where's my *global* variable?
> # Worse, my variable can be gone or not, depending on whether exception
> # triggered or not.
> print(e)
> 
>
> So, perhaps the change should be not removing "e = None" part, but
> conversely, removing the "del e" part.

No, there's a good reason for having the "del e", and that's to
prevent reference loops (since the exception includes the full
traceback, including all local variables). However, I would argue -
and HAVE argued - that this would be a perfect situation for an inner
scope, where the inner variable "e" is actually shadowing, rather than
overwriting, your outer "e". But the push-back against *that* proposal
was insane.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

Hello,

On Sun, 6 Jan 2019 22:19:39 +1100
Chris Angelico  wrote:

> > It's clear that what happens there is that first None is stored to
> > N, just to del it as the next step. Indeed, that's what done in the
> > compile.c:
> >
> > https://github.com/python/cpython/blob/master/Python/compile.c#L2905
> >
> > Storing None looks superfluous.
> >
> > There's no clear explanation why it's done like that, so probably an
> > artifact of the initial implementation.  
> 
> With something as clearly deliberate as this, it's generally safe to
> assume there's a good reason for it.

Absolutely, and one doesn't need to look far for an example. For 20
years Python was devoid of ":=", then, suddenly... ;-).

> But I would very much like to see
> that reason documented, preferably with an example of something that
> could otherwise fail.

Yeah, I'm keen to know too!

[]

> > So, perhaps the change should be not removing "e = None" part, but
> > conversely, removing the "del e" part.  
> 
> No, there's a good reason for having the "del e", and that's to
> prevent reference loops 

So, I'm not sure if I made that clear, but the issue is that currently
the generated code tries to break the reference loop *twice*. But once
should be enough. And which one of "N = None" or "del N" is better is
something to consider.

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Steven D'Aprano

On Sun, Jan 06, 2019 at 01:10:48PM +0200, Paul Sokolovsky wrote:

[...]
> P.S.
> I actually wanted to argue that such an implementation is hardly ideal
> at all. Consider:
> 
> 
> e = "my precious data"
> try:
> 1/0
> except Exception as e:
> pass
> 
> # Where's my *global* variable?
> # Worse, my variable can be gone or not, depending on whether exception
> # triggered or not.
> print(e)

That's not "worse", that's BETTER.

With e deleted, you get an immediate exception the moment you try to use 
it, and you *know* something has gone wrong, instead of the program 
continuing, either to fail much later, where it isn't clear why the code 
fails, or worse, not fail at all, but instead silently do the wrong 
thing.

Incorrect code that fails sooner is better than incorrect code that 
fails later, and MUCH better than incorrect code that doesn't fail at 
all.

The real problem here is that if e holds your precious data, why are you 
assigning an exception to e? You probably wouldn't complain that this 
breaks your program:

e = "my precious data"
import sys as e
# where is my data???

or this:

e = "my precious data"
e = None

so why do you expect `except Exception as e` to be different? Its just a 
name binding operation, the same as all the others.

Regardless of the mechanism, if you don't want to lose your precious 
data, then don't use the same name. But if you truly, really must 
overwrite e, then there is a simple work-around:

e = "my precious data"
try:
1/0
except Exception as err:
e = err

Having the exception name unbound at the end of the block is a weird and 
unfortunate wart on the language, but not as unfortunate as having 
exceptions cause long-lasting reference cycles.

-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

Hello,

On Sun, 6 Jan 2019 23:10:24 +1100
Steven D'Aprano  wrote:

[]

> > # Where's my *global* variable?
> > # Worse, my variable can be gone or not, depending on whether
> > exception # triggered or not.
> > print(e)  
> 
> That's not "worse", that's BETTER.
> 
> With e deleted, you get an immediate exception the moment you try to
> use it, and you *know* something has gone wrong

Ack. Such an explanation makes sense, though semantics behind it is
"magic". And given that variable deletion happens conditionally on
whether exception happened or not, it's still pretty strange. I would
definitely find a behavior of merely clearing an "except" target
variable to be more understandable than magic fiddling with namespaces.

[]

> The real problem here is that if e holds your precious data, why are
> you assigning an exception to e?

Because I got an idea that "except Exc as var" introduces a nested
lexical scope for "var"? I.e., a completely new variable is introduced,
which may shadow, but not override, any on the enclosing scope.

Why would I get that idea? Well, because lexical scoping is a
well-known concept in many programming languages, Python included. For
example, list/set/etc. comprehensions have proper lexical scope, how
would I imagine that specifically in except clauses, it's not a proper
lexical scoping, but a leaky "emulation" of it? 

That said, the description in
https://docs.python.org/3/reference/compound_stmts.html#the-try-statement
doesn't give any false promises regarding lexical scoping. Instead, it
effectively says that the behavior is CPython implementation detail:

"Exceptions are cleared because with the traceback attached to them,
they form a reference cycle"

So, the only thing to take away from this might be indeed an opportunity
to save 4 bytes of bytecode/2 dispatches.

> Having the exception name unbound at the end of the block is a weird
> and unfortunate wart on the language, but not as unfortunate as
> having exceptions cause long-lasting reference cycles.

Thanks, that summarizes it well. And well, my interest is also how
non-compliant would be for another Python implementation to act
differently, specifically to skip wrapping an except handler body in
try-finally (i.e., go back to Python2 behavior). I'm keen to add such
an option to my fork of MicroPython.

> 
> 
> -- 
> Steve

[]

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Steven D'Aprano

On Sun, Jan 06, 2019 at 03:03:03PM +0200, Paul Sokolovsky wrote:

[...]
> Ack. Such an explanation makes sense, though semantics behind it is
> "magic". And given that variable deletion happens conditionally on
> whether exception happened or not, it's still pretty strange.

I don't think anyone *likes* the current behaviour. Speaking for myself, 
I think it is definitely a wart on the language, but a justifiable one, 
since the alternative is worse.

So yes, it is strange, and is something that Python programmers have to 
learn. Probably through experience.

> I would
> definitely find a behavior of merely clearing an "except" target
> variable to be more understandable than magic fiddling with namespaces.

I don't understand what this means. What is "clearing an except target"? 
Do you mean deleting the target name? Names are either bound to an 
object, or they aren't bound at all, so I don't know what it means to 
"clear" a name if it doesn't mean delete it.

> []
> 
> > The real problem here is that if e holds your precious data, why are
> > you assigning an exception to e?
> 
> Because I got an idea that "except Exc as var" introduces a nested
> lexical scope for "var"? I.e., a completely new variable is introduced,
> which may shadow, but not override, any on the enclosing scope.
> 
> Why would I get that idea? Well, because lexical scoping is a
> well-known concept in many programming languages, Python included. For
> example, list/set/etc. comprehensions have proper lexical scope, how
> would I imagine that specifically in except clauses, it's not a proper
> lexical scoping, but a leaky "emulation" of it? 

It isn't an emulation of lexical scope at all. Only the exception target 
is deleted, not other variables bound inside the block.

But a better question is, why would you (generic, not you personally) 
imagine that, alone out of all flow control statements, ONLY "except" 
clauses introduce a new scope? Every other flow control statement (for, 
while, if, elif, else, try, with) runs in the current scope. The only 
statements which create a new scope are def and class. (Did I miss any?)

[...]
> Thanks, that summarizes it well. And well, my interest is also how
> non-compliant would be for another Python implementation to act
> differently, specifically to skip wrapping an except handler body in
> try-finally (i.e., go back to Python2 behavior). I'm keen to add such
> an option to my fork of MicroPython.

Wouldn't that mean that MicroPython suffers from the exception/traceback 
reference cycle problem? How do you propose to solve that?

-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On Mon, Jan 7, 2019 at 1:21 AM Steven D'Aprano  wrote:
> > I would
> > definitely find a behavior of merely clearing an "except" target
> > variable to be more understandable than magic fiddling with namespaces.
>
> I don't understand what this means. What is "clearing an except target"?
> Do you mean deleting the target name? Names are either bound to an
> object, or they aren't bound at all, so I don't know what it means to
> "clear" a name if it doesn't mean delete it.

Setting it to None before unbinding it. It's apparently not enough to
say "del e" - you have to do "e = None; del e". Check the OP.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Serhiy Storchaka

06.01.19 13:10, Paul Sokolovsky пише:

Storing None looks superfluous. For example, DELETE_FAST explicitly
stores NULL on delete.
https://github.com/python/cpython/blob/master/Python/ceval.c#L2249

Likewise, for DELETE_NAME/DELETE_GLOBAL, PyObject_DelItem() is
called which translates to:

m->mp_ass_subscript(o, key, (PyObject*)NULL);

So hopefully any compliant mapping implementation actually clears
stored value, not leaving it dangling.

The "store None" behavior can be traced down to introduction of the
"del except target var" behavior back in 2007:
https://github.com/python/cpython/commit/b940e113bf90ff71b0ef57414ea2beea9d2a4bc0#diff-cb296cc5109f5640ff3f6d7198a6abeeR1999

There's no clear explanation why it's done like that, so probably an
artifact of the initial implementation. Note that even
https://github.com/python/cpython/commit/520b7ae27e39d1c77ea74ccd1b184d7cb43f9dcb
which did quite a bunch of refactoring to "except" implementation, and
reformatted this code, otherwise left it in place.

Because there is a reason for such code.

See issue1631942 [1] and the thread with the subject "self-contained
exceptions" on the Python-3000 mailing list [2] for the rationale.

In short, the code

try:
1/0
except Exception as e:
del e

should work.

I do not see a problem with storing None before deleting a variable.
This is not a performance critical code, because it is executed only
when an exception was raised and caught.

[1] https://bugs.python.org/issue1631942
[2] https://mail.python.org/pipermail/python-3000/2007-January/005294.html

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On 1/6/19 9:14 AM, Steven D'Aprano wrote:
> [...]
> But a better question is, why would you (generic, not you personally) 
> imagine that, alone out of all flow control statements, ONLY "except" 
> clauses introduce a new scope? Every other flow control statement (for, 
> while, if, elif, else, try, with) runs in the current scope. The only 
> statements which create a new scope are def and class. (Did I miss any?)

To be fair except is already unique in that it /does/ "pseudo-scope" the
binding to the variable. The other obvious comparisons are to for loops
and context managers, both of which bind a value to a name that survives
after the exit of the control flow statement.

Given the reference counting reasons for exceptions *not* to outlive
their control flow statement, they are the "odd man out" in that they
delete the exception after the control statement's body exits. To me,
the natural and intuitive way to do this would be to have the exception
live in its own scope where it shadows existing variables, rather than
replacing and then completely removing them. The way it works now is
halfway between the behavior of existing control flow statements and
having a proper nested scope.

Not saying that anything has to change - in the end this is one of the
more minor "gotchas" about Python, and there may be practical reasons
for leaving it as it is - but I do think it's worth noting that for many
people this /will/ be surprising behavior, even if you know that
exceptions don't survive outside of the "except" block.

signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On Mon, Jan 7, 2019 at 2:27 AM Serhiy Storchaka  wrote:
> > The "store None" behavior can be traced down to introduction of the
> > "del except target var" behavior back in 2007:
> > https://github.com/python/cpython/commit/b940e113bf90ff71b0ef57414ea2beea9d2a4bc0#diff-cb296cc5109f5640ff3f6d7198a6abeeR1999
> >
> > There's no clear explanation why it's done like that, so probably an
> > artifact of the initial implementation. Note that even
> > https://github.com/python/cpython/commit/520b7ae27e39d1c77ea74ccd1b184d7cb43f9dcb
> > which did quite a bunch of refactoring to "except" implementation, and
> > reformatted this code, otherwise left it in place.
>
> Because there is a reason for such code.

What reason though??

> See issue1631942 [1] and the thread with the subject "self-contained
> exceptions" on the Python-3000 mailing list [2] for the rationale.
>
> In short, the code
>
>  try:
>  1/0
>  except Exception as e:
>  del e
>
> should work.

But obviously not, because the code was specifically written to set it
to None first. It's not the sort of thing that can be done
accidentally, is it?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Serhiy Storchaka


06.01.19 17:35, Chris Angelico пише:

On Mon, Jan 7, 2019 at 2:27 AM Serhiy Storchaka  wrote:

Because there is a reason for such code.


What reason though??


I added references and excerpts in the previous message.


In short, the code

  try:
  1/0
  except Exception as e:
  del e

should work.


But obviously not, because the code was specifically written to set it
to None first.

Did you tried it? It works.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Return type of datetime subclasses added to timedelta

I can think of many reasons why datetime is different from builtins,
though to be honest I'm not sure that consistency for its own sake is
really a strong argument for keeping a counter-intuitive behavior - and
to be honest I'm open to the idea that /all/ arithmetic types /should/
have some form of this change.

That said, I would say that the biggest difference between datetime and
builtins (other than the fact that datetime is /not/ a builtin, and as
such doesn't necessarily need to be categorized in this group), is that
unlike almost all other arithmetic types, /datetime/ has a special,
dedicated type for describing differences in datetimes. Using your
example of a float subclass, consider that without the behavior of
"addition of floats returns floats", it would be hard to predict what
would happen in this situation:

>>> F(1.2) + 3.4

Would that always return a float, even though F(1.2) + F(3.4) returns an
F? Would that return an F because F is the left-hand operand? Would it
return a float because float is the right-hand operand? Would you walk
the MROs and find the lowest type in common between the operands and
return that? It's not entirely clear which subtype predominates. With
datetime, you have:

datetime - datetime -> timedelta
datetime ± timedelta -> datetime
timedelta ± timedelta -> timedelta

There's no operation between two datetime objects that would return a
datetime object, so it's always clear: operations between datetime
subclasses return timedelta, operations between a datetime object and a
timedelta return the subclass of the datetime that it was added to or
subtracted from.

Of course, the real way to resolve whether datetime should be different
from int/float/string/etc is to look at why this choice was actually
made for those types in the first place, and decide whether datetime is
like them /in this respect/. The heterogeneous operations problem may be
a reasonable justification for leaving the other builtins alone but
changing datetime, but if someone knows of other fundamental reasons why
the decision to have arithmetic operations always create the base class
was chosen, please let me know.

Best,
Paul

On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>
>
> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle  > wrote:
>
> .. the original objection was that this implementation assumes
> that the datetime subclass has a constructor with the same (or a
> sufficiently similar) signature as datetime.
>
> While this was used as a possible rationale for the way standard types
> behave, the main objection to changing datetime classes is that it
> will make them behave differently from builtins.  For example:
>
> >>> class F(float):
> ...     pass
> ...
> >>> type(F.fromhex('AA'))
> 
> >>> type(F(1) + F(2))
> 
>
> This may be a legitimate gripe, but unfortunately that ship has
> sailed long ago. All of datetime's alternate constructors make
> this assumption. Any subclass that does not meet this requirement
> must have worked around it long ago (or they don't care about
> alternate constructors).
>
>
> This is right, but the same argument is equally applicable to int,
> float, etc. subclasses.  If you want to limit your change to datetime
> types you should explain what makes these types special.  

signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On Mon, Jan 7, 2019 at 2:52 AM Serhiy Storchaka  wrote:
>
> 06.01.19 17:35, Chris Angelico пише:
> > On Mon, Jan 7, 2019 at 2:27 AM Serhiy Storchaka  wrote:
> >> Because there is a reason for such code.
> >
> > What reason though??
>
> I added references and excerpts in the previous message.

Which I read, and they do not explain the assignment of None. The only
reference is that the tracker issue mentions having a test case to
ensure that it's happening, which is further proof that it's
intentional, but still fails to explain *why*.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Paul G

I think Serhiy is saying that if you delete exc before the except body ends, 
when the compiled code goes to delete it it will fail. Binding to None 
guarantees that there is something to delete.

On January 6, 2019 4:05:27 PM UTC, Chris Angelico  wrote:
>On Mon, Jan 7, 2019 at 2:52 AM Serhiy Storchaka 
>wrote:
>>
>> 06.01.19 17:35, Chris Angelico пише:
>> > On Mon, Jan 7, 2019 at 2:27 AM Serhiy Storchaka
> wrote:
>> >> Because there is a reason for such code.
>> >
>> > What reason though??
>>
>> I added references and excerpts in the previous message.
>
>Which I read, and they do not explain the assignment of None. The only
>reference is that the tracker issue mentions having a test case to
>ensure that it's happening, which is further proof that it's
>intentional, but still fails to explain *why*.
>
>ChrisA
>___
>Python-Dev mailing list
>Python-Dev@python.org
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe:
>https://mail.python.org/mailman/options/python-dev/paul%40ganssle.io
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On Mon, Jan 7, 2019 at 3:13 AM Paul G  wrote:
>
> I think Serhiy is saying that if you delete exc before the except body ends, 
> when the compiled code goes to delete it it will fail. Binding to None 
> guarantees that there is something to delete.

Ahh! Thank you. I interpreted that example as a "this is what it
compiles to", rather than actual source code that might fail. Can we
get this example, suitably annotated, into the documentation
somewhere?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Paul G

I am not familiar enough with the compiler logic, but would it be possible to 
optimize this away by detecting whether the bound name has already been deleted 
during the body of the except statement?


On January 6, 2019 4:18:00 PM UTC, Chris Angelico  wrote:
>On Mon, Jan 7, 2019 at 3:13 AM Paul G  wrote:
>>
>> I think Serhiy is saying that if you delete exc before the except
>body ends, when the compiled code goes to delete it it will fail.
>Binding to None guarantees that there is something to delete.
>
>Ahh! Thank you. I interpreted that example as a "this is what it
>compiles to", rather than actual source code that might fail. Can we
>get this example, suitably annotated, into the documentation
>somewhere?
>
>ChrisA
>___
>Python-Dev mailing list
>Python-Dev@python.org
>https://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe:
>https://mail.python.org/mailman/options/python-dev/paul%40ganssle.io
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Guido van Rossum

On Sun, Jan 6, 2019 at 9:40 AM Paul G  wrote:

> I am not familiar enough with the compiler logic, but would it be possible
> to optimize this away by detecting whether the bound name has already been
> deleted during the body of the except statement?
>

As was said before, it's not useful to optimize code that only runs on an
exception. (Though if there was a

On January 6, 2019 4:18:00 PM UTC, Chris Angelico  wrote:
>
> On Mon, Jan 7, 2019 at 3:13 AM Paul G  wrote:
>
>>
>>  I think Serhiy is saying that if you delete exc before the except body 
>> ends, when the compiled code goes to delete it it will fail. Binding to None 
>> guarantees that there is something to delete.
>>
>
> Ahh! Thank you. I interpreted that example as a "this is what it
> compiles to", rather than actual source code that might fail. Can we
> get this example, suitably annotated, into the documentation
> somewhere?
>
>
I don't see a reason to document this behavior -- it would be strange if it
*didn't* work.

If we had an opcode to delete a variable but didn't raise if it is unset,
we could use that, but otherwise I see no action needed here.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Return type of datetime subclasses added to timedelta

2019-01-06 Thread Andrew Svetlov

>From my perspective datetime classes are even more complex than int/float.
Let's assume we have

class DT(datetime.datetime): ...
class TD(datetime.timedelta): ...

What is the result type for the following expressions?
DT - datetime
DT - DT
DT + TD
DT + timedelta

I have a feeling that the question has no generic answer.
For *particular* implementation you can override all __add__, __sub__
and other arithmetic operations, and you can do it right now with the
current datetime module implementation.

P.S.
I think inheritance from datetime classes is a very rare thing, 99.99% of
users don't need it.


On Sun, Jan 6, 2019 at 6:03 PM Paul Ganssle  wrote:

> I can think of many reasons why datetime is different from builtins,
> though to be honest I'm not sure that consistency for its own sake is
> really a strong argument for keeping a counter-intuitive behavior - and to
> be honest I'm open to the idea that *all* arithmetic types *should* have
> some form of this change.
>
> That said, I would say that the biggest difference between datetime and
> builtins (other than the fact that datetime is *not* a builtin, and as
> such doesn't necessarily need to be categorized in this group), is that
> unlike almost all other arithmetic types, *datetime* has a special,
> dedicated type for describing differences in datetimes. Using your example
> of a float subclass, consider that without the behavior of "addition of
> floats returns floats", it would be hard to predict what would happen in
> this situation:
>
> >>> F(1.2) + 3.4
>
> Would that always return a float, even though F(1.2) + F(3.4) returns an
> F? Would that return an F because F is the left-hand operand? Would it
> return a float because float is the right-hand operand? Would you walk the
> MROs and find the lowest type in common between the operands and return
> that? It's not entirely clear which subtype predominates. With datetime,
> you have:
>
> datetime - datetime -> timedelta
> datetime ± timedelta -> datetime
> timedelta ± timedelta -> timedelta
>
> There's no operation between two datetime objects that would return a
> datetime object, so it's always clear: operations between datetime
> subclasses return timedelta, operations between a datetime object and a
> timedelta return the subclass of the datetime that it was added to or
> subtracted from.
>
> Of course, the real way to resolve whether datetime should be different
> from int/float/string/etc is to look at why this choice was actually made
> for those types in the first place, and decide whether datetime is like
> them *in this respect*. The heterogeneous operations problem may be a
> reasonable justification for leaving the other builtins alone but changing
> datetime, but if someone knows of other fundamental reasons why the
> decision to have arithmetic operations always create the base class was
> chosen, please let me know.
>
> Best,
> Paul
> On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>
>
>
> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle  wrote:
>
>> .. the original objection was that this implementation assumes that the
>> datetime subclass has a constructor with the same (or a sufficiently
>> similar) signature as datetime.
>>
> While this was used as a possible rationale for the way standard types
> behave, the main objection to changing datetime classes is that it will
> make them behave differently from builtins.  For example:
>
> >>> class F(float):
> ... pass
> ...
> >>> type(F.fromhex('AA'))
> 
> >>> type(F(1) + F(2))
> 
>
> This may be a legitimate gripe, but unfortunately that ship has sailed
>> long ago. All of datetime's alternate constructors make this assumption.
>> Any subclass that does not meet this requirement must have worked around it
>> long ago (or they don't care about alternate constructors).
>>
>
> This is right, but the same argument is equally applicable to int, float,
> etc. subclasses.  If you want to limit your change to datetime types you
> should explain what makes these types special.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Return type of datetime subclasses added to timedelta

2019-01-06 Thread Guido van Rossum

I don't think datetime and builtins like int necessarily need to be
aligned. But I do see a problem -- the __new__ and __init__ methods defined
in the subclass (if any) should allow for being called with the same
signature as the base datetime class. Currently you can have a subclass of
datetime whose __new__ has no arguments (or, more realistically, interprets
its arguments differently). Instances of such a class can still be added to
a timedelta. The proposal would cause this to break (since such an addition
has to create a new instance, which calls __new__ and __init__). Since this
is a backwards incompatibility, I don't see how it can be done -- and I
also don't see many use cases, so I think it's not worth pursuing further.

Note that the same problem already happens with the .fromordinal() class
method, though it doesn't happen with .fromdatetime() or .now():

>>> class D(datetime.datetime):
...   def __new__(cls): return cls.now()
...
>>> D()
D(2019, 1, 6, 10, 33, 37, 161606)
>>> D.fromordinal(100)
Traceback (most recent call last):
  File "", line 1, in 
TypeError: __new__() takes 1 positional argument but 4 were given
>>> D.fromtimestamp(123456789)
D(1973, 11, 29, 13, 33, 9)
>>>

On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle  wrote:

> I can think of many reasons why datetime is different from builtins,
> though to be honest I'm not sure that consistency for its own sake is
> really a strong argument for keeping a counter-intuitive behavior - and to
> be honest I'm open to the idea that *all* arithmetic types *should* have
> some form of this change.
>
> That said, I would say that the biggest difference between datetime and
> builtins (other than the fact that datetime is *not* a builtin, and as
> such doesn't necessarily need to be categorized in this group), is that
> unlike almost all other arithmetic types, *datetime* has a special,
> dedicated type for describing differences in datetimes. Using your example
> of a float subclass, consider that without the behavior of "addition of
> floats returns floats", it would be hard to predict what would happen in
> this situation:
>
> >>> F(1.2) + 3.4
>
> Would that always return a float, even though F(1.2) + F(3.4) returns an
> F? Would that return an F because F is the left-hand operand? Would it
> return a float because float is the right-hand operand? Would you walk the
> MROs and find the lowest type in common between the operands and return
> that? It's not entirely clear which subtype predominates. With datetime,
> you have:
>
> datetime - datetime -> timedelta
> datetime ± timedelta -> datetime
> timedelta ± timedelta -> timedelta
>
> There's no operation between two datetime objects that would return a
> datetime object, so it's always clear: operations between datetime
> subclasses return timedelta, operations between a datetime object and a
> timedelta return the subclass of the datetime that it was added to or
> subtracted from.
>
> Of course, the real way to resolve whether datetime should be different
> from int/float/string/etc is to look at why this choice was actually made
> for those types in the first place, and decide whether datetime is like
> them *in this respect*. The heterogeneous operations problem may be a
> reasonable justification for leaving the other builtins alone but changing
> datetime, but if someone knows of other fundamental reasons why the
> decision to have arithmetic operations always create the base class was
> chosen, please let me know.
>
> Best,
> Paul
> On 1/5/19 3:55 AM, Alexander Belopolsky wrote:
>
>
>
> On Wed, Jan 2, 2019 at 10:18 PM Paul Ganssle  wrote:
>
>> .. the original objection was that this implementation assumes that the
>> datetime subclass has a constructor with the same (or a sufficiently
>> similar) signature as datetime.
>>
> While this was used as a possible rationale for the way standard types
> behave, the main objection to changing datetime classes is that it will
> make them behave differently from builtins.  For example:
>
> >>> class F(float):
> ... pass
> ...
> >>> type(F.fromhex('AA'))
> 
> >>> type(F(1) + F(2))
> 
>
> This may be a legitimate gripe, but unfortunately that ship has sailed
>> long ago. All of datetime's alternate constructors make this assumption.
>> Any subclass that does not meet this requirement must have worked around it
>> long ago (or they don't care about alternate constructors).
>>
>
> This is right, but the same argument is equally applicable to int, float,
> etc. subclasses.  If you want to limit your change to datetime types you
> should explain what makes these types special.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.o

Re: [Python-Dev] Return type of datetime subclasses added to timedelta

On 1/6/19 1:29 PM, Andrew Svetlov wrote:
> From my perspective datetime classes are even more complex than int/float.
> Let's assume we have
>
> class DT(datetime.datetime): ...
> class TD(datetime.timedelta): ...
>
> What is the result type for the following expressions?
> DT - datetime
> DT - DT
> DT + TD
> DT + timedelta
>
It is not really complicated, the default "difference between two
datetimes" returns a `timedelta`, you can change that by overriding
`__sub__` or `__rsub__` as desired, but there's no reason to think that
the fact that just because DT is a subclass of datetime that it would be
coupled to a specific timedelta subclass *by default*.

Similarly, DT + TD by default will do whatever "datetime" and
"timedelta" do unless you specifically override them. In my proposal,
adding some time to a datetime subclass would return an object of the
datetime subclass, so unless __radd__ or __rsub__ were overriden in
`timedelta`, that's what would happen, the defaults would be (sensibly):

DT - datetime -> timedelta
DT - DT -> timedelta
DT + TD -> DT
DT + timedelta -> timedelta

The only time it would be more complicated is if datetime were defined
like this:

class datetime:
    TIMEDELTA_CLASS = datetime.timedelta
    ...

In which case you'd have the same problem you have with float/int/etc
(not a particularly more complicated one. But that's not the case, and
there /is/ one obviously right answer. This is not the case with float
subclasses, because the intuitive rule is "adding together two objects
of the same class gives the same class", which fails when you have two
different subclasses. With datetime, you have "adding a delta type to a
value type returns an object of the value type", which makes perfect
sense, as opposed to "adding a delta type to a value type returns the
base value type, even if the base value type was never used".

> I have a feeling that the question has no generic answer.
> For *particular* implementation you can override all __add__, __sub__
> and other arithmetic operations, and you can do it right now with the
> current datetime module implementation.
> P.S.
> I think inheritance from datetime classes is a very rare thing, 99.99%
> of users don't need it.
>
Both of these points are addressed in my original post, IIRC, but both
of these arguments cut both ways. Assuming it's true that this is very
rare - the 0.01% of people who /are/ subclassing datetime either don't
care about this behavior or want timedelta arithmetic to return their
subclass. It's rare enough that there should be no problem giving them
what they want.

Similarly, the rarest group - people who are creating datetime
subclasses /and/ want the original behavior - can simply implement
__add__ and __sub__ to get what they want, so there's no real conflict,
it's just a matter of setting a sane default that also solves the
problem that datetime alternate constructors tend to leak their
implementation details because of the arithmetic return type issue.

Best, Paul

signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Serhiy Storchaka


06.01.19 18:05, Chris Angelico пише:

Which I read, and they do not explain the assignment of None. The only
reference is that the tracker issue mentions having a test case to
ensure that it's happening, which is further proof that it's
intentional, but still fails to explain *why*.


This is explained on a discussion on the Python-3000 mailing list. I 
provided a code example which would fail if do not perform such 
assignment. The test, mentioned in the tracker issue, contains similar code.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

Hello,

On Sun, 6 Jan 2019 17:26:09 +0200
Serhiy Storchaka  wrote:

[]

> Because there is a reason for such code.
> 
> See issue1631942 [1] and the thread with the subject "self-contained 
> exceptions" on the Python-3000 mailing list [2] for the rationale.
> 
> In short, the code
> 
>  try:
>  1/0
>  except Exception as e:
>  del e
> 
> should work.

(Dark) Magic. Live and learn/think. So, we allow user to delete a var,
then recreate it, just to delete again. Thanks for both the hint and
references, Serhiy!

I can only agree with what Chris wrote in the initial reply - it
would nice if there would be an explanation of these tricks somewhere.
I myself wouldn't know a better place than the source code comments.
Well, at least I did trace it to the source code where it's handled.

> I do not see a problem with storing None before deleting a variable. 
> This is not a performance critical code, because it is executed only 
> when an exception was raised and caught.

>From CPython's points of view, I might (well, have to) agree. But I was
pretty surprised/disappointed that MicroPython follows the same
routine, I'm glad I now understand ins and outs of the choices made.

> 
> [1] https://bugs.python.org/issue1631942
> [2]
> https://mail.python.org/pipermail/python-3000/2007-January/005294.html


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Return type of datetime subclasses added to timedelta

I did address this in the original post - the assumption that the
subclass constructor will have the same arguments as the base
constructor is baked into many alternate constructors of datetime. I
acknowledge that this is a breaking change, but it is a small one -
anyone creating such a subclass that /cannot/ handled the class being
created this way would be broken in myriad ways.

We have also in recent years changed several alternate constructors
(including `replace`) to retain the original subclass, which by your
same standard would be a breaking change. I believe there have been no
complaints. In fact, between Python 3.6 and 3.7, the very example you
showed broke:

Python 3.6.6:

>>> class D(datetime.datetime):
... def __new__(cls):
... return cls.now()
...
>>> D()
D(2019, 1, 6, 13, 49, 38, 842033)

Python 3.7.2:

>>> class D(datetime.datetime):
... def __new__(cls):
... return cls.now()
...
>>> D()
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in __new__
TypeError: __new__() takes 1 positional argument but 9 were given

We haven't seen any bug reports about this sort of thing; what we /have/
been getting is bug reports that subclassing datetime doesn't retain the
subclass in various ways (because people /are/ using datetime
subclasses). This is likely to cause very little in the way of problems,
but it will improve convenience for people making datetime subclasses
and almost certainly performance for people using them (e.g. pendulum
and arrow, which now need to take a slow pure python route in many
situations to work around this problem).

If we're /really/ concerned with this backward compatibility breaking,
we could do the equivalent of:

try:
    return new_behavior(...)
except TypeError:
    warnings.warn("The semantics of timedelta addition have "
  "changed in a way that raises an error in "
  "this subclass. Please implement __add__ "
  "if you need the old behavior.", DeprecationWarning)

Then after a suitable notice period drop the warning and turn it to a
hard error.

Best,

Paul

On 1/6/19 1:43 PM, Guido van Rossum wrote:
> I don't think datetime and builtins like int necessarily need to be
> aligned. But I do see a problem -- the __new__ and __init__ methods
> defined in the subclass (if any) should allow for being called with
> the same signature as the base datetime class. Currently you can have
> a subclass of datetime whose __new__ has no arguments (or, more
> realistically, interprets its arguments differently). Instances of
> such a class can still be added to a timedelta. The proposal would
> cause this to break (since such an addition has to create a new
> instance, which calls __new__ and __init__). Since this is a backwards
> incompatibility, I don't see how it can be done -- and I also don't
> see many use cases, so I think it's not worth pursuing further.
>
> Note that the same problem already happens with the .fromordinal()
> class method, though it doesn't happen with .fromdatetime() or .now():
>
> >>> class D(datetime.datetime):
> ...   def __new__(cls): return cls.now()
> ...
> >>> D()
> D(2019, 1, 6, 10, 33, 37, 161606)
> >>> D.fromordinal(100)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: __new__() takes 1 positional argument but 4 were given
> >>> D.fromtimestamp(123456789)
> D(1973, 11, 29, 13, 33, 9)
> >>>
>
> On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle  > wrote:
>
> I can think of many reasons why datetime is different from
> builtins, though to be honest I'm not sure that consistency for
> its own sake is really a strong argument for keeping a
> counter-intuitive behavior - and to be honest I'm open to the idea
> that /all/ arithmetic types /should/ have some form of this change.
>
> That said, I would say that the biggest difference between
> datetime and builtins (other than the fact that datetime is /not/
> a builtin, and as such doesn't necessarily need to be categorized
> in this group), is that unlike almost all other arithmetic types,
> /datetime/ has a special, dedicated type for describing
> differences in datetimes. Using your example of a float subclass,
> consider that without the behavior of "addition of floats returns
> floats", it would be hard to predict what would happen in this
> situation:
>
> >>> F(1.2) + 3.4
>
> Would that always return a float, even though F(1.2) + F(3.4)
> returns an F? Would that return an F because F is the left-hand
> operand? Would it return a float because float is the right-hand
> operand? Would you walk the MROs and find the lowest type in
> common between the operands and return that? It's not entirely
> clear which subtype predominates. With datetime, you have:
>
> datetime - datetime -> timedelta
> datetime ± timedelta -> datetime
> timedelta ± timedelta -> timedelta
>

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

Hello,

On Mon, 7 Jan 2019 01:14:55 +1100
Steven D'Aprano  wrote:

[]

> > Thanks, that summarizes it well. And well, my interest is also how
> > non-compliant would be for another Python implementation to act
> > differently, specifically to skip wrapping an except handler body in
> > try-finally (i.e., go back to Python2 behavior). I'm keen to add
> > such an option to my fork of MicroPython.  
> 
> Wouldn't that mean that MicroPython suffers from the
> exception/traceback reference cycle problem? How do you propose to
> solve that?

To answer this, MicroPython doesn't have reference counting and relies
solely on garbage collection, so breaking cycles early wouldn't have
much effect. Breaking stray links to should-be-dead data structures
might (to allow them to be collected sooner rather than later), but
probably not at the expense of introducing extra entry in the block
stack. (That may seems like breadcrumb saving, but saving a few
percents of overhead in many places is how MicroPython achieves
its promise of programming small embedded systems in a very high level
language.)

[]

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

2019-01-06 Thread Brett Cannon

On Sun, 6 Jan 2019 at 10:26, Guido van Rossum  wrote:

> On Sun, Jan 6, 2019 at 9:40 AM Paul G  wrote:
>
>> I am not familiar enough with the compiler logic, but would it be
>> possible to optimize this away by detecting whether the bound name has
>> already been deleted during the body of the except statement?
>>
>
> As was said before, it's not useful to optimize code that only runs on an
> exception. (Though if there was a
>
> On January 6, 2019 4:18:00 PM UTC, Chris Angelico 
> wrote:
>>
>> On Mon, Jan 7, 2019 at 3:13 AM Paul G  wrote:
>>
>>>
>>>  I think Serhiy is saying that if you delete exc before the except body 
>>> ends, when the compiled code goes to delete it it will fail. Binding to 
>>> None guarantees that there is something to delete.
>>>
>>
>> Ahh! Thank you. I interpreted that example as a "this is what it
>> compiles to", rather than actual source code that might fail. Can we
>> get this example, suitably annotated, into the documentation
>> somewhere?
>>
>>
> I don't see a reason to document this behavior -- it would be strange if
> it *didn't* work.
>
> If we had an opcode to delete a variable but didn't raise if it is unset,
> we could use that, but otherwise I see no action needed here.
>

Maybe someone can propose a code comment to explain this instead? I agree
that this doesn't need to be explained in the language spec, but I don't
see any harm in a comment where the opcodes are emitted to explain why it
is the way it is .
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Compilation of "except FooExc as var" adds useless store

On Mon, Jan 7, 2019 at 7:18 AM Brett Cannon  wrote:
>
> Maybe someone can propose a code comment to explain this instead? I agree 
> that this doesn't need to be explained in the language spec, but I don't see 
> any harm in a comment where the opcodes are emitted to explain why it is the 
> way it is .
>

No BPO as yet but here's a tracker issue to get started.

https://github.com/python/cpython/pull/11448

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Return type of datetime subclasses added to timedelta

2019-01-06 Thread Brett Cannon

On Sun, 6 Jan 2019 at 11:00, Paul Ganssle  wrote:

> I did address this in the original post - the assumption that the subclass
> constructor will have the same arguments as the base constructor is baked
> into many alternate constructors of datetime. I acknowledge that this is a
> breaking change, but it is a small one - anyone creating such a subclass
> that *cannot* handled the class being created this way would be broken in
> myriad ways.
>
> We have also in recent years changed several alternate constructors
> (including `replace`) to retain the original subclass, which by your same
> standard would be a breaking change. I believe there have been no
> complaints. In fact, between Python 3.6 and 3.7, the very example you
> showed broke:
>
> Python 3.6.6:
>
> >>> class D(datetime.datetime):
> ... def __new__(cls):
> ... return cls.now()
> ...
> >>> D()
> D(2019, 1, 6, 13, 49, 38, 842033)
>
> Python 3.7.2:
>
> >>> class D(datetime.datetime):
> ... def __new__(cls):
> ... return cls.now()
> ...
> >>> D()
> Traceback (most recent call last):
>   File "", line 1, in 
>   File "", line 3, in __new__
> TypeError: __new__() takes 1 positional argument but 9 were given
>
>
> We haven't seen any bug reports about this sort of thing; what we *have*
> been getting is bug reports that subclassing datetime doesn't retain the
> subclass in various ways (because people *are* using datetime subclasses).
>

To help set expectations, the current semantics are not a bug and so the
proposal isn't fixing a bug but proposing a change in semantics.


> This is likely to cause very little in the way of problems, but it will
> improve convenience for people making datetime subclasses and almost
> certainly performance for people using them (e.g. pendulum and arrow, which
> now need to take a slow pure python route in many situations to work around
> this problem).
>
> If we're *really* concerned with this backward compatibility breaking,
>

We very much do care. Because this isn't a bug but a voluntary semantic
change you're proposing to change we can't blindly break people who are
relying on the current semantics. We need to have a justification for those
people as to why we have decided to change the semantics now after all of
these years as well as provide an upgrade path.

-Brett


> we could do the equivalent of:
>
> try:
> return new_behavior(...)
> except TypeError:
> warnings.warn("The semantics of timedelta addition have "
>   "changed in a way that raises an error in "
>   "this subclass. Please implement __add__ "
>   "if you need the old behavior.", DeprecationWarning)
>

> Then after a suitable notice period drop the warning and turn it to a hard
> error.
>
> Best,
>
> Paul
> On 1/6/19 1:43 PM, Guido van Rossum wrote:
>
> I don't think datetime and builtins like int necessarily need to be
> aligned. But I do see a problem -- the __new__ and __init__ methods defined
> in the subclass (if any) should allow for being called with the same
> signature as the base datetime class. Currently you can have a subclass of
> datetime whose __new__ has no arguments (or, more realistically, interprets
> its arguments differently). Instances of such a class can still be added to
> a timedelta. The proposal would cause this to break (since such an addition
> has to create a new instance, which calls __new__ and __init__). Since this
> is a backwards incompatibility, I don't see how it can be done -- and I
> also don't see many use cases, so I think it's not worth pursuing further.
>
> Note that the same problem already happens with the .fromordinal() class
> method, though it doesn't happen with .fromdatetime() or .now():
>
> >>> class D(datetime.datetime):
> ...   def __new__(cls): return cls.now()
> ...
> >>> D()
> D(2019, 1, 6, 10, 33, 37, 161606)
> >>> D.fromordinal(100)
> Traceback (most recent call last):
>   File "", line 1, in 
> TypeError: __new__() takes 1 positional argument but 4 were given
> >>> D.fromtimestamp(123456789)
> D(1973, 11, 29, 13, 33, 9)
> >>>
>
> On Sun, Jan 6, 2019 at 9:05 AM Paul Ganssle  wrote:
>
>> I can think of many reasons why datetime is different from builtins,
>> though to be honest I'm not sure that consistency for its own sake is
>> really a strong argument for keeping a counter-intuitive behavior - and to
>> be honest I'm open to the idea that *all* arithmetic types *should* have
>> some form of this change.
>>
>> That said, I would say that the biggest difference between datetime and
>> builtins (other than the fact that datetime is *not* a builtin, and as
>> such doesn't necessarily need to be categorized in this group), is that
>> unlike almost all other arithmetic types, *datetime* has a special,
>> dedicated type for describing differences in datetimes. Using your example
>> of a float subclass, consider that without the behavior of "addition of
>> floats returns floats", it would be hard to predict what woul

Re: [Python-Dev] Return type of datetime subclasses added to timedelta