[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Victor Stinner]

On Sat, Feb 6, 2021 at 3:26 PM Inada Naoki  wrote:
> I changed my mind. Since there is no plan to change the default
> encoding for now,
> no need to encourage `encoding="locale"` soon.
>
> Until users can drop Python 3.9 support, they can use EncodingWarning
> only for finding missing `encoding="utf-8"` or `encoding="ascii"`.
>
> I will remove the io.LOCALE_ENCODING.

I recall that something like 1 year ago, I basically tried to
implement something like your PEP, to see if the stdlib calls open()
without specifying an encoding. There were so many warnings, that the
output was barely readable.

The warning would only be useful if there is a way to modify the code
to make the warning quiet (fix the issue) without losing support with
Python 3.9 and older.

I understand that open(filename) must be replaced with open(filename,
encoding=("locale" if sys.version_info >= (3, 10) else None)) to make
it backward and forward compatibility without emitting an
EncodingWarning. One issue is that some people may blindly copy/paste
this code pattern without thinking if "locale" is the proper encoding.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6CRWIH6AJ43H2IRQZDJFUSSYUFPDSY3L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Signature discrepancies between documentation and implementation

[Erlend Aasland]

Hi all,

What's the recommended approach with issues like 
https://bugs.python.org/issue43094? Change the docs or the implementation? I 
did a quick search on bpo, but could not find similar past issues.



Erlend
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3NPQMMWIDSP5DEBV5LAQ3Y7JUIUN4PFE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Victor Stinner]

If there are applications relying on the parameter name, it's better
to trust the Python implementation, rather than the documentation.

It would be better to make these parameters positional only in the
first place, but now it's too late for such backward incompatible
change :-(

Victor

On Tue, Feb 9, 2021 at 11:43 AM Erlend Aasland  wrote:
>
> Hi all,
>
> What's the recommended approach with issues like 
> https://bugs.python.org/issue43094? Change the docs or the implementation? I 
> did a quick search on bpo, but could not find similar past issues.
>
>
>
> Erlend
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/3NPQMMWIDSP5DEBV5LAQ3Y7JUIUN4PFE/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/X6SZIVOZ233TLLJV43UQEHMV3ELGP34S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Erlend Aasland]

Hi Victor, and thanks for you reply.

> On 9 Feb 2021, at 11:46, Victor Stinner  wrote:
> If there are applications relying on the parameter name, it's better
> to trust the Python implementation, rather than the documentation.

I feared that answer :)

> It would be better to make these parameters positional only in the
> first place, but now it's too late for such backward incompatible
> change :-(

Unfortunately, yes.


Erlend
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7XJMO335TVWAUZWXLOH6272E5K5FJL7N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Inada Naoki]

On Tue, Feb 9, 2021 at 7:23 PM Victor Stinner  wrote:
>
> I recall that something like 1 year ago, I basically tried to
> implement something like your PEP, to see if the stdlib calls open()
> without specifying an encoding. There were so many warnings, that the
> output was barely readable.
>
> The warning would only be useful if there is a way to modify the code
> to make the warning quiet (fix the issue) without losing support with
> Python 3.9 and older.
>
> I understand that open(filename) must be replaced with open(filename,
> encoding=("locale" if sys.version_info >= (3, 10) else None)) to make
> it backward and forward compatibility without emitting an
> EncodingWarning.

I think most of them must be replaced with encoding="ascii" or encoding="utf-8".

And encoding=locale.getpreferredencoding(False) is backward/forward
compatible way.
There is very little difference between encoding=None and
encoding=locale.getpreferredencoding(False).
But it is not a problem for most use cases.
Only applications using PYTHONLEGACYWINDOWSSTDIO and open() for
console I/O are affected by difference between them.


> One issue is that some people may blindly copy/paste
> this code pattern without thinking if "locale" is the proper encoding.
>

Isn't it same if the code pattern become `encoding=getattr(io,
"LOCALE_ENCODING", None)`,
or `encoding=locale.getpreferredencoding(False)`?

I think only we can do is documenting the option like this:

"""
EncodingWarning is warning to find missing encoding="utf-8" option. It
is common pitfall that many Windows user
Don't try to fix them if you need to use locale specific encoding.
"""

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YLAC2WJZ2TX7I3I6TSWA4GWPP5NNETUH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Paul Moore]

On Tue, 9 Feb 2021 at 11:55, Inada Naoki  wrote:

> I think only we can do is documenting the option like this:
>
> """
> EncodingWarning is warning to find missing encoding="utf-8" option. It
> is common pitfall that many Windows user
> Don't try to fix them if you need to use locale specific encoding.
> """

I'm a very strong -1 on having programs generate warnings that the
user isn't supposed to fix. If we can't provide a good recommendation
to the user on what to do, we shouldn't be warning them that what they
are currently doing is wrong. I've seen far too many examples of
people thinking "well, users can ignore the warning, it's not shown by
default" and then users' code being broken because of a situation we
didn't think about (most recently, the Python test suite, which runs
the venv tests with warnings converted to errors, which broke on a pip
release that contains a deprecation warning from packaging).

IMO, if we issue a warning, we *must* be able to advise the user how
to fix it. Otherwise we shouldn't be assuming we know what's correct
better than the user.

Personally, I'm not at all keen on the idea of making users always
specify encoding in the first place, even if it's "just for the
transition". There are far too many people in my experience who
wouldn't have a clue what to do when faced with that decision. And the
people (again in my experience) who don't know how to make that choice
are *precisely* the people for whom the system-defined default is what
they want. Certainly, if they are getting stuff off the internet, they
will more often get UTF-8, but I tend to find that people with limited
understanding of these issues are much more comfortable with the idea
that "stuff off the internet needs weird settings like this UTF-8
thing whatever it is", than they are with the idea that they have to
tell Python how to read that text file they just got from their boss,
who's still got Windows 7 on his PC...

If we want to switch the default encoding from the locale encoding to
UTF-8, we should find a way to do that which *doesn't* mean that
there's a "transitional" state where using the default is considered
bad practice. That helps no-one, and just adds confusion, which will
last far longer than that one release (there will be people
encountering StackOverflow questions on the topic long after the
default has changed).

Maybe we just have to accept that we can't work out what people are
intending, and just state in advance in the documentation that the
default will change, then it's documented as an upcoming breaking
change that people can address (if they read the release notes, but we
seem to be assuming they'll spot a warning, so why not assume they
read the release notes, too?).

Paul

PS I've hesitated about saying this before, as I'm very aware that
being from the UK, any problems I have with encodings are relatively
minor, so I want to let the people with real problems have their say.
But when we're talking about telling users not to fix warnings, I feel
the need to speak up.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5W7YFNY7BLCS25ZICWMH57XH5REITI34/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Victor Stinner]

On Tue, Feb 9, 2021 at 1:31 PM Paul Moore  wrote:
> If we can't provide a good recommendation
> to the user on what to do, we shouldn't be warning them that what they
> are currently doing is wrong.

The warning can explicitly suggest to use encoding="utf8", it should
work in almost all cases.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NMHJAPSNE7XI65DKV6EB55HGQW5XRAA6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Paul Moore]

But people who currently don't specify the encoding, and *don't* have
any issue (because the system locale is correct) will be getting told
to introduce a bug into their code, if they follow that advice :-(

Paul

On Tue, 9 Feb 2021 at 16:03, Victor Stinner  wrote:
>
> On Tue, Feb 9, 2021 at 1:31 PM Paul Moore  wrote:
> > If we can't provide a good recommendation
> > to the user on what to do, we shouldn't be warning them that what they
> > are currently doing is wrong.
>
> The warning can explicitly suggest to use encoding="utf8", it should
> work in almost all cases.
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HSV6QSJKAUFS7LWZVEZUWTUD5A6DCFFL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] PEP 647 (type guards) -- final call for comments

[Guido van Rossum]

I think we have reached consensus on PEP 647 in typing-sig. We have
implementations for mypy and pyright, not sure about the rest. This PEP
does not affect CPython directly except for the addition of one special
item (TypeGuard) to typing.py -- it would be nice to get that in the 3.10
stdlib.

I'm CC'ing python-dev here to see if there are any further comments; if
not, we can initiate the approval process by creating an issue at
https://github.com/python/steering-council.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NOLCFYLYAQQHXISNMPYCEOAZ7ZPFCGUW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Inada Naoki]

On Wed, Feb 10, 2021 at 1:19 AM Paul Moore  wrote:
>
> But people who currently don't specify the encoding, and *don't* have
> any issue (because the system locale is correct) will be getting told
> to introduce a bug into their code, if they follow that advice :-(
>

This warning is opt-in warning like BytesWarning.

It will be a good tool to find potential problems for people knows
what is the problem.
But it is not recommended for users who don't understand what is the problem.

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SJKTVKW3DQCPRFRTGOUL73EI6BOGWDFF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Paul Moore]

On Tue, 9 Feb 2021 at 16:28, Inada Naoki  wrote:
>
> On Wed, Feb 10, 2021 at 1:19 AM Paul Moore  wrote:
> >
> > But people who currently don't specify the encoding, and *don't* have
> > any issue (because the system locale is correct) will be getting told
> > to introduce a bug into their code, if they follow that advice :-(
> >
>
> This warning is opt-in warning like BytesWarning.
>
> It will be a good tool to find potential problems for people knows
> what is the problem.
> But it is not recommended for users who don't understand what is the problem.

Ah, OK. I missed that point in the long email chain. Sorry.
Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UUX2IR655Y6JOCOQBPHHQTPUUTVFA5XA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Anders Munch]

Victor Stinner [mailto:vstin...@python.org] wrote:
> The warning can explicitly suggest to use encoding="utf8", it should work in 
> almost all cases.

The warning should also explain how to get backwards-compatible behaviour, i.e. 
suggest encoding="locale". 

Inada Naoki   wrote:
> This warning is opt-in warning like BytesWarning.

What use is a warning that no-one sees?  When the default is switched to 
encoding="utf8", it will break software, and people need to be warned of that.
UnicodeDecodeError's will abound when files that used to be read in a 
single-byte encoding fails to decode as utf-8. All it takes is a single é.
If the default encoding is ever to change, there's no way around a noisy 
warning.

How about swapping around "locale" and None?  That is, make "locale" the new 
default that emits a warning, and encoding=None emits no warning.  That has the 
advantage that old code can be updated to say encoding=None, and then it will 
work on both old and new Pythons without warning.

regards, Anders
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GZOHZAXKJDRJPF32U2ET5E32SOYXHR5E/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Inada Naoki]

On Tue, Feb 9, 2021 at 9:31 PM Paul Moore  wrote:
>
> Personally, I'm not at all keen on the idea of making users always
> specify encoding in the first place, even if it's "just for the
> transition".

I agree with you. But as I wrote in the PEP, omitted encoding caused
much troubles already.
Windows users can not just `pip install somepkg` because some library
authors write `long_description=open("README.md").read()` in setup.py.

I am trying to fix this situation by two parallel approaches:

* (This PEP) Provide a tool for finding this type of bugs, and
recommend `encoding="utf-8"` for cross-platform library authors.
* (Author thread) Make UTF-8 mode more usable for Windows users,
especially students.


> If we want to switch the default encoding from the locale encoding to
> UTF-8, we should find a way to do that which *doesn't* mean that
> there's a "transitional" state where using the default is considered
> bad practice. That helps no-one, and just adds confusion, which will
> last far longer than that one release (there will be people
> encountering StackOverflow questions on the topic long after the
> default has changed).
>
> Maybe we just have to accept that we can't work out what people are
> intending, and just state in advance in the documentation that the
> default will change, then it's documented as an upcoming breaking
> change that people can address (if they read the release notes, but we
> seem to be assuming they'll spot a warning, so why not assume they
> read the release notes, too?).
>

This PEP doesn't cover how to change the default encoding. So this is
slightly off topic.
I have two ideas for changing the default encoding:

(a) Regular deprecation period: Emitting EncodingWarning by default
(3.14 or later), and change the default encoding later (3.17 or
later).
(b) Enable UTF-8 mode default on Windows. Users can disable UTF-8 mode
for backward compatibility.

Steve Dower againsted to (b) very strongly. He suggested to emit
DeprecationWarning.
https://discuss.python.org/t/pep-597-enable-utf-8-mode-by-default-on-windows/3122/16

On the other hand, some core-dev don't like emitting Warning for all
omitted `encoding` option.

So I don't have strong opinion about which approach is better. I want
to see how EncodingWarning and UTF-8 mode are adopted.

I want to implement both EncodingWarning and per-site UTF-8 mode
setting in Python 3.10.
5+ years later, we will see which approach is adopted by users.

* If EncodingWarning is widely adopted by many developers, we can
discuss approach (a).
* If UTF-8 mode becomes the best practice for Windows users, we can
discuss approach (b).

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DY4OPCBKHHRJZMXEJ43MXPNXJ4EUS6MM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Paul Moore]

On Tue, 9 Feb 2021 at 16:52, Anders Munch  wrote:
> How about swapping around "locale" and None?  That is, make "locale" the new 
> default that emits a warning, and encoding=None emits no warning.  That has 
> the advantage that old code can be updated to say encoding=None, and then it 
> will work on both old and new Pythons without warning.

I don't understand why working code should have to change *twice*. I'm
fine with the idea that people *actually* relying on the current
default will need to switch when the default changes, but making them
change once to silence the warning and then again to explicitly select
the old default is pretty annoying.

If we don't want people to use the default encoding, we should just
make encoding a required argument and stop pretending. If omitting the
encoding and using the default is intended to be a supported usage,
then we should *not* penalise people doing that. Changing the default
is a backward-incompatible change, that's enough of an inconvenience.
Changing the (behaviour of the) default *twice* is just making things
worse.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/36HJRYU6R6NEDZY7QSKS3DEKRY6OLTI4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Inada Naoki]

On Wed, Feb 10, 2021 at 1:46 AM Anders Munch  wrote:
>
>
> Inada Naoki   wrote:
> > This warning is opt-in warning like BytesWarning.
>
> What use is a warning that no-one sees?

At least, I see.
We can fix stdlib and tests first, and fix some major tools too.

After that, `encoding="locale"` becomes backward/forward compatible at
some point.

> When the default is switched to encoding="utf8", it will break software, and 
> people need to be warned of that.
> UnicodeDecodeError's will abound when files that used to be read in a 
> single-byte encoding fails to decode as utf-8. All it takes is a single é.
> If the default encoding is ever to change, there's no way around a noisy 
> warning.
>

Please read the PEP and some my posts in this threads.
We are not discussing about changing default encoding for now.

This PEP provides a tool to find missing `encoding="utf-8"` bug for now.
The goal of the PEP is encourage `encoding="utf-8"` when the user
assumes encoding is UTF-8.

If we decide to change the default encoding. EncodingWarning can be
used to discourage omitting the `encoding` option.
But it is out of scope of the PEP. We don't discourage omitting
encoding option in Python 3.10.


> How about swapping around "locale" and None?  That is, make "locale" the new 
> default that emits a warning, and encoding=None emits no warning.  That has 
> the advantage that old code can be updated to say encoding=None, and then it 
> will work on both old and new Pythons without warning.
>

I thought it, but it may not work. Consider about function like this:

```
def read_text(self, encoding=None):
with open(self._filename, encoding=encoding) as f:
return f.read()
```

If `encoding=None` suppresses the warning, functions like this never warned.

So I think current PEP is better.
If users want to use locale encoding, they don't need to fix the
warning anytime soon. They can wait to drop Python 3.9 support.
If they want to fix all warnings soon, they can
`encoding=locale.getpreferredencoding(False)`.

Regards,
-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4Q74PW673RMBMQTDZXHTVE6X7FT6DSAL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Victor Stinner]

On Tue, Feb 9, 2021 at 5:51 PM Anders Munch  wrote:
> Victor Stinner [mailto:vstin...@python.org] wrote:
> > The warning can explicitly suggest to use encoding="utf8", it should work 
> > in almost all cases.
>
> The warning should also explain how to get backwards-compatible behaviour, 
> i.e. suggest encoding="locale".

encoding="utf8" is backward compatible and is likely to fix encoding
bugs when the locale encoding is not UTF-8. It is likely what the
developer expected, without knowing that open(filename) does not
always use UTF-8. See PEP 597 rationale.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L2H34EXNE7XUQX3XILLPDNGOQHVK6ENR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] python limited API missing PyCMethod_New on Windows for 3.9 and 3.10

[Barry Scott]

I raised https://bugs.python.org/issue43155 for the missing
PyCMethod_New that was added in 3.9 but is missing from the
python3.lib.

Barry
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5AGJXVX7JCA2ZP5ZFUWBECGFKGYFKXOD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Jakub Stasiak]

> On 9 Feb 2021, at 11:22, Erlend Aasland  wrote:
> 
> Hi all,
> 
> What's the recommended approach with issues like 
> https://bugs.python.org/issue43094? Change the docs or the implementation? I 
> did a quick search on bpo, but could not find similar past issues.
> 
> 
> 
> Erlend

To provide some context as you looked for issues on bpo: in the two I was 
involved in[1][2] the resolution was to update the documenttion to reflect the 
actual situation.

Best,
Jakub

[1] https://bugs.python.org/issue42230
[2] https://bugs.python.org/issue38580
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FHWE3YHZNZE7QRTSH5FA4Z7A2LQUBROB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 647 (type guards) -- final call for comments

[Terry Reedy]

On 2/9/2021 11:21 AM, Guido van Rossum wrote:
I think we have reached consensus on PEP 647 in typing-sig. We have 
implementations for mypy and pyright, not sure about the rest. This PEP 
does not affect CPython directly except for the addition of one special 
item (TypeGuard) to typing.py -- it would be nice to get that in the 
3.10 stdlib.


I'm CC'ing python-dev here to see if there are any further comments;


As a naive possible future user of typing, I have a couple of related 
comments on the presentation and suggested additions with regard to the 
type of TypeGuard 'instances' and the meaning of type guard function 
returns.


"Its return type would be changed from bool to TypeGuard[List[str]]."

My first thought was "Huh? The return type *is* a bool!"  Then I 
realized that a TypeGuard[List[str]] is a 'specified affirmative bool', 
a bool for which True means 'is a List[str]'.  More generally, a 
TypeGuard[X] is a bool for which True affirmatively means that the first 
argument of the function is specifically an X.  I mentally condensed 
that to 'bool subtype' and suspect that others have and will do the 
same, but you later reject 'subtype' because 'subtype' has a specific 
meaning in the typing world that does not apply here.


Suggestion 1:  Rather than mere say what a TypeGuard is not, follow the 
quoted sentence with a statement of what it is, such as I gave above.


"Specification\n TypeGuard Type \n "

The first paragraph specifies that the function must be a predicate 
(returns bool and only bool).  It leaves out that it must be a positive 
or affirmative predicate for which 'True' says what the first argument 
is, rather than what it is not.  The examples meet this requirement and 
narrowing only for 'if' clauses implies this requirement, but ...


Suggestion 2: End the first paragraph by stating the requirement.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MF2DRQ3GIBNGFOUDWL4ZSVPBCQPVYVQO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Serhiy Storchaka]

09.02.21 12:22, Erlend Aasland пише:
> What's the recommended approach with issues like 
> https://bugs.python.org/issue43094? Change the docs or the implementation? I 
> did a quick search on bpo, but could not find similar past issues.

If the documentation and the C implemented function contradict about
parameter name, we are free to treat the parameter as positional-only.
User cannot pass the argument as keyword because the documented name
does not work, and the real name is not exposed to the user.

In this case, there are two similar functions, create_function() and
create_aggregate(). Both are documented as having parameter
"num_params", but real names are different: "narg" and "n_arg". It would
be confusing to have different spelling of the same parameter in similar
functions. I am sure that it would be difficult to remember how is it
spelled in every case. Also, in Python terminology, the semantic of this
name is the number of parameters, not the number of argument.

So I think that in this particular case the chance of breaking some code
is insignificant, but possible long-term harm of exposing bad parameter
names may be significant.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JE2E5RJZQPMXHTWIU3NA74YDMEZHGYUK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Brett Cannon]

On Tue., Feb. 9, 2021, 12:20 Serhiy Storchaka,  wrote:

> 09.02.21 12:22, Erlend Aasland пише:
> > What's the recommended approach with issues like
> https://bugs.python.org/issue43094? Change the docs or the
> implementation? I did a quick search on bpo, but could not find similar
> past issues.
>
> If the documentation and the C implemented function contradict about
> parameter name, we are free to treat the parameter as positional-only.
> User cannot pass the argument as keyword because the documented name
> does not work, and the real name is not exposed to the user.
>

I agree with Serhiy's logic.

-Brett


> In this case, there are two similar functions, create_function() and
> create_aggregate(). Both are documented as having parameter
> "num_params", but real names are different: "narg" and "n_arg". It would
> be confusing to have different spelling of the same parameter in similar
> functions. I am sure that it would be difficult to remember how is it
> spelled in every case. Also, in Python terminology, the semantic of this
> name is the number of parameters, not the number of argument.
>
> So I think that in this particular case the chance of breaking some code
> is insignificant, but possible long-term harm of exposing bad parameter
> names may be significant.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JE2E5RJZQPMXHTWIU3NA74YDMEZHGYUK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L5KVZMBF6VDLA42KRODHCGNKIGGEMIVL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Paul Moore]

On Tue, 9 Feb 2021 at 16:54, Inada Naoki  wrote:
>
> On Tue, Feb 9, 2021 at 9:31 PM Paul Moore  wrote:
> >
> > Personally, I'm not at all keen on the idea of making users always
> > specify encoding in the first place, even if it's "just for the
> > transition".
>
> I agree with you. But as I wrote in the PEP, omitted encoding caused
> much troubles already.
> Windows users can not just `pip install somepkg` because some library
> authors write `long_description=open("README.md").read()` in setup.py.
>
> I am trying to fix this situation by two parallel approaches:
>
> * (This PEP) Provide a tool for finding this type of bugs, and
> recommend `encoding="utf-8"` for cross-platform library authors.
> * (Author thread) Make UTF-8 mode more usable for Windows users,
> especially students.

Thanks for explaining (again). There's so much debate, across multiple
proposals, that I can barely follow it. I'm impressed that you're
managing to keep things straight at all :-)

I guess my views on this PEP come down to

* I see no harm in having a tool that helps developers spot
platform-specific assumptions about encoding.
* Realistically, I'd be surprised if developers actually use such a
tool. If they were likely to do so, they could probably just as easily
locate all the uses of open() in their code, and check that way. So
I'm not sure this proposal is actually worth it, even if the end
result would be very beneficial.
* In the setup.py case, why don't those same Windows users complain
that the library fails to install? A quick bug report, followed by a
simple fix, seems more likely to happen than the developer suddenly
deciding to scan their code for encoding issues.

Regarding the wider question of UTF8 as default, my views can probably
be summarised as follows:

* If you want to write correct code to deal with encodings, there is
no substitute for carefully considering every bytes/string conversion,
deciding how you are going to identify the encoding to use, and then
specifying that encoding explicitly. Default values for encodings have
no place in such code.
* In reality, though, that's far too much work for many situations.
Default encodings are a necessary convenience, particularly for simple
scripts, or for people who can't, or don't want to, do the analysis
that the "correct" approach implies.
* Picking the right default is *hard*. Changing the default is even
harder, unfortunately.
* I feel that we already have a number of mechanisms (PEPs 538 and
540) trying to tackle this issue. Adding yet more suggests to me that
we'd be better off pausing and working out why we still have an issue.
We should be moving towards *fewer* mechanisms, not more.
* We have UTF-8 mode, and users can set it per-process (via flag or
environment variable) per-user or per-site (by environment variable).
I don't honestly believe that a user (whatever OS they work on) who is
capable of writing Python code, can't be shown how to set an
environment variable. I see no reason to suggest we need yet another
way to set UTF-8 mode, or that a per-interpreter or per-virtualenv
setting is particularly crucial (suggestions that have been made in
the Python-Ideas threads).
* UTF-8 is likely to be the most appropriate default encoding for
Python in the longer term, and I agree that Windows is fast
approaching the point where a UTF-8 encoding is more appropriate than
the ANSI codepage for "new stuff". But there's a lot of legacy files
and applications around, and I suspect that a UTF-8 default will
inconvenience a lot of people working with such data. But equally,
such people may not be in a huge rush to switch to the latest Python
version. Whichever way we go, though, some people will be
inconvenienced.

I'm also somewhat bemused by the rather negative view of "Windows
beginners" that lies behind a lot of these discussions. People's
experiences may well differ, but the people I see using (and learning)
Python on Windows are often experienced computer users, maybe
developers with significant experience in Java or other "enterprise
languages", or data scientists who have a lot of knowledge of
computers, but are relatively new to programming. Or systems admins,
or database specialists, who want to use Python to write scripts on
Windows. None of those people fit the picture of people who wouldn't
know how to set an environment variable, or configure their
environment. On the other hand, (in my experience) they often don't
really have much knowledge of character encodings, and tend to just
use whatever default their PC uses, and expect it to work. They *can*,
however, understand when an encoding problem is explained to them, and
can set an explicit encoding once they know they need to.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/a

[Python-Dev] Re: Signature discrepancies between documentation and implementation

[Petr Viktorin]

On 2/9/21 9:15 PM, Serhiy Storchaka wrote:

09.02.21 12:22, Erlend Aasland пише:

What's the recommended approach with issues like 
https://bugs.python.org/issue43094? Change the docs or the implementation? I 
did a quick search on bpo, but could not find similar past issues.


If the documentation and the C implemented function contradict about
parameter name, we are free to treat the parameter as positional-only.
User cannot pass the argument as keyword because the documented name
does not work, and the real name is not exposed to the user.


It is. Python will correct you if you try to use the documented name:

>>> import sqlite3
>>> connection = sqlite3.connect(':memory:')
>>> connection.create_function('testfunc', num_params=1, func=lambda 
arg: None)

Traceback (most recent call last):
  File "", line 1, in 
TypeError: function missing required argument 'narg' (pos 2)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UHDNIUTPS6FESF2VRK5AJMHUZIDKZL2N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Inada Naoki]

On Wed, Feb 10, 2021 at 5:50 AM Paul Moore  wrote:
>
> On Tue, 9 Feb 2021 at 16:54, Inada Naoki  wrote:
> >
> > On Tue, Feb 9, 2021 at 9:31 PM Paul Moore  wrote:
> > >
> > > Personally, I'm not at all keen on the idea of making users always
> > > specify encoding in the first place, even if it's "just for the
> > > transition".
> >
> > I agree with you. But as I wrote in the PEP, omitted encoding caused
> > much troubles already.
> > Windows users can not just `pip install somepkg` because some library
> > authors write `long_description=open("README.md").read()` in setup.py.
> >
> > I am trying to fix this situation by two parallel approaches:
> >
> > * (This PEP) Provide a tool for finding this type of bugs, and
> > recommend `encoding="utf-8"` for cross-platform library authors.
> > * (Author thread) Make UTF-8 mode more usable for Windows users,
> > especially students.
>
> Thanks for explaining (again). There's so much debate, across multiple
> proposals, that I can barely follow it. I'm impressed that you're
> managing to keep things straight at all :-)
>
> I guess my views on this PEP come down to
>
> * I see no harm in having a tool that helps developers spot
> platform-specific assumptions about encoding.
> * Realistically, I'd be surprised if developers actually use such a
> tool. If they were likely to do so, they could probably just as easily
> locate all the uses of open() in their code, and check that way. So
> I'm not sure this proposal is actually worth it, even if the end
> result would be very beneficial.
> * In the setup.py case, why don't those same Windows users complain
> that the library fails to install? A quick bug report, followed by a
> simple fix, seems more likely to happen than the developer suddenly
> deciding to scan their code for encoding issues.
>

Yes, some issues are solved already.
On the other hand, there are dozen question about UnicodeDecodeError
in Q&A sites like Stack Overflow.
Many people don't know what the error means, and how to report it correctly.

I sometime set PYTHONWARNINGS=deafult in my bashrc and find
DeprecationWarnings in libraries I am using, and report them.

On the other hand, it is difficult to find omitted `encoding="utf-8"`,
because I use macOS and Linux in daily development.
If there is PYTHONWARNENCODING, I can write `export
PYTHONWARNENCODING=1` in my .bashrc.


> Regarding the wider question of UTF8 as default, my views can probably
> be summarised as follows:
>
> * If you want to write correct code to deal with encodings, there is
> no substitute for carefully considering every bytes/string conversion,
> deciding how you are going to identify the encoding to use, and then
> specifying that encoding explicitly. Default values for encodings have
> no place in such code.
> * In reality, though, that's far too much work for many situations.
> Default encodings are a necessary convenience, particularly for simple
> scripts, or for people who can't, or don't want to, do the analysis
> that the "correct" approach implies.

Yes. and the UTF-8 is the default encoding for s.encode() already.

> * Picking the right default is *hard*. Changing the default is even
> harder, unfortunately.
> * I feel that we already have a number of mechanisms (PEPs 538 and
> 540) trying to tackle this issue. Adding yet more suggests to me that
> we'd be better off pausing and working out why we still have an issue.
> We should be moving towards *fewer* mechanisms, not more.
> * We have UTF-8 mode, and users can set it per-process (via flag or
> environment variable) per-user or per-site (by environment variable).
> I don't honestly believe that a user (whatever OS they work on) who is
> capable of writing Python code, can't be shown how to set an
> environment variable. I see no reason to suggest we need yet another
> way to set UTF-8 mode, or that a per-interpreter or per-virtualenv
> setting is particularly crucial (suggestions that have been made in
> the Python-Ideas threads).

Note that many Python users don't use consoles. They just starts
Jupyter Notebook, or they just write .py file and run it in the
Minecraft mods.

> * UTF-8 is likely to be the most appropriate default encoding for
> Python in the longer term, and I agree that Windows is fast
> approaching the point where a UTF-8 encoding is more appropriate than
> the ANSI codepage for "new stuff". But there's a lot of legacy files
> and applications around, and I suspect that a UTF-8 default will
> inconvenience a lot of people working with such data. But equally,
> such people may not be in a huge rush to switch to the latest Python
> version. Whichever way we go, though, some people will be
> inconvenienced.
>
> I'm also somewhat bemused by the rather negative view of "Windows
> beginners" that lies behind a lot of these discussions. People's
> experiences may well differ, but the people I see using (and learning)
> Python on Windows are often experienced computer users, maybe
> developers with significan

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Terry Reedy]

On 2/9/2021 8:28 PM, Inada Naoki wrote:


Note that many Python users don't use consoles.


Those of use who do may find it hard to imagine just how easy we have 
made computing.


My daughter minored in Computer Science about 6 years ago.  She never 
saw Command Prompt until the summer after her Junior year when she 
watched me use it to install numpy and other packages for her.  I had to 
do it because 'Run pip install numpy', etc, was met with a blank stare. 
 I had taught her Python with IDLE, downloaded and install with a 
browser, and had neglected to teach her 'Dos' until then.


So had her CS classes.  Those previous used Racket in a Dr. something 
environment and Java in, I believe, Eclipse.  Also downloaded and 
installed with a browser.



They just starts
Jupyter Notebook, or they just write .py file and run it in the
Minecraft mods.


Also similar.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O5SS7I2H2GRCM2YBOVYX7CWUYTX2J7AO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

[Jonathan Goble]

On Tue, Feb 9, 2021 at 11:29 PM Terry Reedy  wrote:
>
> On 2/9/2021 8:28 PM, Inada Naoki wrote:
>
> > Note that many Python users don't use consoles.
>
> Those of use who do may find it hard to imagine just how easy we have
> made computing.
>
> My daughter minored in Computer Science about 6 years ago.  She never
> saw Command Prompt until the summer after her Junior year when she
> watched me use it to install numpy and other packages for her.  I had to
> do it because 'Run pip install numpy', etc, was met with a blank stare.
>   I had taught her Python with IDLE, downloaded and install with a
> browser, and had neglected to teach her 'Dos' until then.
>
> So had her CS classes.  Those previous used Racket in a Dr. something
> environment and Java in, I believe, Eclipse.  Also downloaded and
> installed with a browser.

Speaking as a current CS undergraduate student here (senior graduating
in December 2021). At my university, the freshman/sophomore-level
programming classes do not assume or expect any type of command line
knowledge. They all rely on GUI tools (Eclipse, IntelliJ, or NetBeans
for the freshman Java courses, Visual Studio for Data Structures in
C++).

There is one course, typically taken in either the second or third
semester for traditional students, called Operating Systems Concepts
and Usage, that broadly discusses how operating systems function, but
is also designed as a first introduction to Linux and to the command
line. (Until this point, the only operating system students are
assumed to be familiar with is Windows.) For many students, this
course is their first ever exposure to the command prompt.

After that, students in this program don't generally *need* to touch
the command line again in their studies until they hit 4000-level
courses, and even then only a few courses require it. Outside of that
one introductory course, I've only had two courses so far that
actually required command line usage. Everything else so far has
offered GUI options, even many upper level courses.

I think it's a disservice to fail to expose students to the command
line more and earlier, but the fact is, that failure happens and
happens often, and developers need to be conscious of that.

Despite my own ease and comfort with the command-line (which dates
back to learning my way around DOS at the age of 5) to the point of
almost always having a terminal window open on my daily Debian
machine, I frequently find myself opting for point-and-click solutions
to common problems, even Git operations (which are so easy and
powerful in VS Code with the GitLens extension). GUI tools grow more
powerful by the day, and it's very easy to get deep into a computer
science program these days and not be comfortable with the command
line and/or not know how to change environment variables.

Python, as a common introductory language used by many thousands of
people who have never taken a university computer course, never mind
majoring in computer science, shouldn't have basic features that
depend on the likely false assumption that the user has ever seen a
command prompt or an environment variable, much less comprehend how to
use them.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UKGADFE6OK66BNUPE36NVHXBZZSOR7OD/
Code of Conduct: http://python.org/psf/codeofconduct/