[Python-Dev] Re: Implementation of PEP-0604

[Ivan Levkivskyi]

On Thu, 31 Oct 2019 at 23:20, Nick Coghlan  wrote:

> [...snip...]
>
>>
>> It's not enough to move the typing classes, I must move
>> functools.lru_cache() and dependencies, collections.abs.Mapping and
>> dependencies, and track the frame level.
>>
>> *It's too big for me.*
>>
>>
> It's certainly not an easy project to tackle.
>
> For some of the specific points you raise though, you wouldn't translate
> the existing Python code directly to C.
>

I agree with Nick, this is indeed big, but not impossible. If you are not
sure yet whether you will work on implementation, you can focus on
polishing the PEP text, and then if it is accepted and you will decide to
give implementation to someone else, we will find a volunteer.

--
Ivan
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AKXBKSBYESEJVKAAPCXHG4Z5EWOS54MM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: A much better tokenize.untokenize function

[Edward K. Ream]

On Sun, Nov 3, 2019 at 7:23 PM Edward K. Ream  wrote:

Clearly, the new code would have to be repackaged if it were to be made
> part of tokenize.py. That's all I would like to say now regarding your
> comments. Perhaps other devs will pick up the ball.
>

After sleeping on your comments, I would like to distill them to their gist:

1. untokenize does not take a contents argument.
2. untokenize does not do exactly the same thing as the new code.

These are legitimate complaints, and I'm not quite sure what, if anything,
can be done about them.

However, I am not quite ready dismiss the possibility of using the the new
code in the tokenize module, because...

*Motivations*

For convenience, lets call the proposed new code the *gem *:-)

Let's be clear, there is no reason for python libraries to include every
little code gem. However, I have two motivations adding the gem to the
tokenize module:

1. The gem would be useful for any token-based beautification code, such as
the token-based versions of black or fstringify that Leo uses. Furthermore,
the present untokenize method teaches away
 from
the gem.

2.  issue 12691  is troubling, because
the unit tests failed to detect a code blunder in add_whitespace.

*Strengthening unit tests*

It's not clear how the gem could strengthen unit tests. Indeed, the gem is
always going to pass ;-)

This is frustrating, because full sources *are *available to TestRoundtrip
*.*check_roundtrip.  Indeed, that method starts with:

if isinstance(f, str):
code = f.encode('utf-8')
else:
code = f.read()
f.close()

If f is a str, then f *is* "contents".  Otherwise, "contents" can be
recreated from the file's bytes.

*Summary*

Your objections are largely valid.  Nevertheless, the gem may be worth
further consideration.

There is no need to add every little code gem to python's standard library.
In this case, there are two reasons why adding the gem to tokenize.py might
be a good idea:

1. The gem would be of use to anyone writing a token-based tool. Such tools
*do* have access to full sources.

2. The present untokenize method teaches away from the gem. The gem
corrects the mistaken impression that untokenizing in the presence of full
sources is difficult.  In fact, it's a snap.

It is an open question whether it might be possible to strengthen the unit
tests for the tokenize module using the gem. Full source*s are* available
to TestRoundtrip*.*check_roundtrip.

Edward
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/S3SHIMJU4ICG7Q6GS2GT36UG4AB5DOWC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: A much better tokenize.untokenize function

[Ethan Furman]

On 11/04/2019 03:42 AM, Edward K. Ream wrote:

On Sun, Nov 3, 2019 at 7:23 PM Edward K. Ream wrote:



Let's be clear, there is no reason for python libraries to include every little 
code gem. However, I have two motivations adding the gem to the tokenize module:


This is all irrelevant if you haven't signed the CLA*.

--
~Ethan~


* https://www.python.org/psf/contrib/contrib-form/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F6MZYCE3FRHAIBFCSMVQ2JTKQDRO5TAJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: A much better tokenize.untokenize function

[Edward K. Ream]

On Mon, Nov 4, 2019 at 5:42 AM Edward K. Ream  wrote:

> ...there are two reasons why adding the gem to tokenize.py might be a
good idea.

Heh.  I myself am not convinced that the gem needs "advertising" in
tokenize.py.

Those interested in how Leo's token-based commands work would be more
likely to find the gem in Leo's Untokenize class.

Edward
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QJK7DYWZY5MMGL526YCXRJEX423M2EBY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: A much better tokenize.untokenize function

[Kyle Stanley]

> This is all irrelevant if you haven't signed the CLA*.

IMO, this shouldn't be an argument against the proposed changes, as the CLA
is fairly straightforward and only takes 24-48 hours to process. Unless the
OP specifically is unable to or doesn't want to sign the CLA, it won't be a
significant concern. Most new authors usually open their first PR without
having the CLA already signed, then have the CLA signed and processed while
waiting for review from a core developer.

On Mon, Nov 4, 2019 at 11:48 AM Ethan Furman  wrote:

> On 11/04/2019 03:42 AM, Edward K. Ream wrote:
> > On Sun, Nov 3, 2019 at 7:23 PM Edward K. Ream wrote:
>
> > Let's be clear, there is no reason for python libraries to include every
> little code gem. However, I have two motivations adding the gem to the
> tokenize module:
>
> This is all irrelevant if you haven't signed the CLA*.
>
> --
> ~Ethan~
>
>
> * https://www.python.org/psf/contrib/contrib-form/
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/F6MZYCE3FRHAIBFCSMVQ2JTKQDRO5TAJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3ZC5VFJGGJBJE467HQ5E3KX346EC6OEN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Implementation of PEP-0604

[Kyle Stanley]

> I agree with Nick, this is indeed big, but not impossible. If you are not
sure yet whether you will work on implementation, you can focus on
polishing the PEP text, and then if it is accepted and you will decide to
give implementation to someone else, we will find a volunteer.

Unless I'm misunderstanding something, it seems that the OP is under the
impression that because they authored the PEP, they're obligated and
responsible for the entire implementation. Instead, we should be able to
relatively easily split up the implementation into multiple self-contained
PRs. The OP can still work on some of them if they want to and are able to,
but other contributors can work on other parts of it.

The easiest way of doing this would likely be separating each
self-contained PR into a bullet point in a bpo issue with a brief
description (similar to the layout posted above), and then the OP can work
through them as they have time to do so. Any other contributor can simply
pick up where the OP left off, or on other parts of it while the OP is
working on something else. This would of course be after/if the PEP is
accepted.

On Mon, Nov 4, 2019 at 7:33 AM Ivan Levkivskyi  wrote:

> On Thu, 31 Oct 2019 at 23:20, Nick Coghlan  wrote:
>
>> [...snip...]
>>
>>>
>>> It's not enough to move the typing classes, I must move
>>> functools.lru_cache() and dependencies, collections.abs.Mapping and
>>> dependencies, and track the frame level.
>>>
>>> *It's too big for me.*
>>>
>>>
>> It's certainly not an easy project to tackle.
>>
>> For some of the specific points you raise though, you wouldn't translate
>> the existing Python code directly to C.
>>
>
> I agree with Nick, this is indeed big, but not impossible. If you are not
> sure yet whether you will work on implementation, you can focus on
> polishing the PEP text, and then if it is accepted and you will decide to
> give implementation to someone else, we will find a volunteer.
>
> --
> Ivan
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/AKXBKSBYESEJVKAAPCXHG4Z5EWOS54MM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YYAKCVANFGRJAVGML4WJ225M5ANYZFDD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do not fallback to __trunc__ when convert to int

[Victor Stinner]

Le jeu. 31 oct. 2019 à 11:41, Serhiy Storchaka  a écrit :
> I propose to deprecate the falling back to __trunc__ for converting to
> int and remove it in future.

As Nick already said, I'm working on PEP 608 "Coordinated Python
release" which discuss such change:
https://www.python.org/dev/peps/pep-0608/

It's discussed at:
https://discuss.python.org/t/rfc-pep-608-coordinated-python-release/2539

Do you have an idea of how many projects would emit such warning? More
and more projects are running their test suite using -Werror. Any new
warning can make a project test suite to fail.

I'm looking for help to build a tool to test popular PyPI projects on
a modified Python version, to be able to answer to such questions.

My current experimental project can only test numpy, coverage and
jinja2, and it doesn't work well (especially on Python 3.9):
https://github.com/vstinner/pythonci/

Since this tool doesn't exist yet and since we cannot test all PyPI
projects, I suggest to keep the option of reverting such change if we
consider that it breaks too many projects during the Python 3.9 beta
phase.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KAIJFQW6OVMRYUIHL7H4KRGF2E2S6WTM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Do not fallback to __trunc__ when convert to int

[Paul G]

> Do you have an idea of how many projects would emit such warning? More
and more projects are running their test suite using -Werror. Any new
warning can make a project test suite to fail.

I don't have an opinion on __trunc__, but as someone who runs such a project, I 
have to say this is exactly the point. I always have the option of adding a 
given warning to my ignore list (as long as it's raised in such a way that this 
is easy to do), and I want the hard error so I can heed the warnings.

Avoiding raising warnings because too many test suites break when using -Werror 
seems counterproductive.

Of course, "lots of stuff breaks with -Werror" could be a proxy for "it would 
be a lot of work to fix this", and maybe all that work isn't worth it for 
whatever improvement you'd get from the fixes, but I think it would be better 
to try and evaluate it on those terms rather than considering broken test 
suites an inherently bad thing.


On November 5, 2019 1:06:42 AM UTC, Victor Stinner  wrote:
>Le jeu. 31 oct. 2019 à 11:41, Serhiy Storchaka  a
>écrit :
>> I propose to deprecate the falling back to __trunc__ for converting
>to
>> int and remove it in future.
>
>As Nick already said, I'm working on PEP 608 "Coordinated Python
>release" which discuss such change:
>https://www.python.org/dev/peps/pep-0608/
>
>It's discussed at:
>https://discuss.python.org/t/rfc-pep-608-coordinated-python-release/2539
>
>Do you have an idea of how many projects would emit such warning? More
>and more projects are running their test suite using -Werror. Any new
>warning can make a project test suite to fail.
>
>I'm looking for help to build a tool to test popular PyPI projects on
>a modified Python version, to be able to answer to such questions.
>
>My current experimental project can only test numpy, coverage and
>jinja2, and it doesn't work well (especially on Python 3.9):
>https://github.com/vstinner/pythonci/
>
>Since this tool doesn't exist yet and since we cannot test all PyPI
>projects, I suggest to keep the option of reverting such change if we
>consider that it breaks too many projects during the Python 3.9 beta
>phase.
>
>Victor
>-- 
>Night gathers, and now my watch begins. It shall not end until my
>death.
>___
>Python-Dev mailing list -- python-dev@python.org
>To unsubscribe send an email to python-dev-le...@python.org
>https://mail.python.org/mailman3/lists/python-dev.python.org/
>Message archived at
>https://mail.python.org/archives/list/python-dev@python.org/message/KAIJFQW6OVMRYUIHL7H4KRGF2E2S6WTM/
>Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PFPNUZIDZ4CW5ODKTS3QINZDY4SROWUY/
Code of Conduct: http://python.org/psf/codeofconduct/