[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, 10 Feb 2021 at 01:29, Inada Naoki wrote: > Note that many Python users don't use consoles. I never suggested that they did. There's a GUI for setting user-level and system-level environment variables. And whoever is introducing the user to Python can show them how to set the necessary environment variable - or do it for them. Please be clear, I'm not saying I don't understand the difficulties. But I do question why PYTHONUTF8 is so much different than all the other environment variables that Python responds to that it needs special additional options. Remember - I've already said that I'm not convinced that setting UTF8 mode globally is the right approach. So what you're saying in effect is that you want to convince me that we should add a new mechanism to globally set an option that I don't believe should be set globally. Let's just assume until you can convince me that setting UTF-8 mode globally is a good idea, there's no point trying to convince me that we need a new mechanism to do so because environment variables aren't good enough. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4UHPNJVJ3ACKREUH7WLEAD5ZB4IR5R2K/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, Feb 10, 2021 at 5:00 PM Paul Moore wrote: > > Let's just assume until you can convince me that setting UTF-8 mode > globally is a good idea, Oh, you misunderstood me. My proposal is not setting UTF-8 mode globally. What I proposed is setting UTF-8 mode per env (e.g. installation, venv, or conda env). But this is off topic. The thread for this topic is here. https://mail.python.org/archives/list/python-id...@python.org/thread/LQVK2UKPSOI2AHYFUWK6ZII2U6QKK6BP/ Regards, -- Inada Naoki ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZXPBI3WSZ6FCAWWKXNBRNKYXUXUG5FEH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Signature discrepancies between documentation and implementation
Thanks, Jakub. E > On 9 Feb 2021, at 19:42, Jakub Stasiak wrote: > > To provide some context as you looked for issues on bpo: in the two I was > involved in[1][2] the resolution was to update the documenttion to reflect > the actual situation. > > Best, > Jakub > > [1] https://bugs.python.org/issue42230 > [2] https://bugs.python.org/issue38580 > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UJJ7CKQ6QYYVLBCUJPT4TUSTYTWFADDT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Signature discrepancies between documentation and implementation
> On 10 Feb 2021, at 01:28, Petr Viktorin wrote: > > On 2/9/21 9:15 PM, Serhiy Storchaka wrote: >> 09.02.21 12:22, Erlend Aasland пише: >>> What's the recommended approach with issues like >>> https://bugs.python.org/issue43094? Change the docs or the implementation? >>> I did a quick search on bpo, but could not find similar past issues. >> If the documentation and the C implemented function contradict about >> parameter name, we are free to treat the parameter as positional-only. >> User cannot pass the argument as keyword because the documented name >> does not work, and the real name is not exposed to the user. > > It is. Python will correct you if you try to use the documented name: Correct. However, I suspect that most users use positional arguments with these functions (since this is the first (?) report of the inconsistency) so I guess it's ok, in this case, to make the arguments positional only and normalise the argument naming. Erlend ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4U3IYJE5ZAXJQWHPXCQZC2K4OYDZWVFE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
Victor Stinner [mailto:vstin...@python.org] wrote:
> encoding="utf8" is backward compatible and is likely to fix encoding bugs
> when the locale encoding is not UTF-8
This program runs just fine on 3.8.7 Windows, against a file.txt that contains
latin-1 text:
with open('file.txt', 'rt') as f:
print(f.read())
But if I change it to this:
with open('file.txt', 'rt', encoding='utf-8') as f:
print(f.read())
then it fails with UnicodeDecodeError. How it that backwards compatible?
regards, Anders
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/SDFLXIW64ESKDBARCHC2A2JA4NFPBZ2Y/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, Feb 10, 2021 at 11:14 PM Anders Munch wrote:
>
> This program runs just fine on 3.8.7 Windows, against a file.txt that
> contains latin-1 text:
>
> with open('file.txt', 'rt') as f:
> print(f.read())
>
> But if I change it to this:
>
> with open('file.txt', 'rt', encoding='utf-8') as f:
> print(f.read())
>
> then it fails with UnicodeDecodeError. How it that backwards compatible?
>
There are several ways:
* encoding="latin1" -- This is the best. Works perfectly.
* Don't touch -- You don't need to enable EncodingWarning.
* encoding=locale.getpreferredencoding(False) -- Backward compatible.
But doesn't work if you enabled UTF-8 mode.
* encoding="mbcs" -- Backward compatible. Works even when you enabled
UTF-8 mode. But it doesn't work only on Windows.
Regards,
--
Inada Naoki
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/XPBVG5GU37UDQPDTZIFIGI2WOFYHYQBU/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Tue, 9 Feb 2021 at 16:52, Anders Munch wrote: >> How about swapping around "locale" and None? That is, make "locale" the new >> default that emits a warning, and encoding=None emits no warning. That has >> the advantage that old code can be updated to say encoding=None, and then it >> will work on both old and new Pythons without warning. Paul Moore [mailto:p.f.mo...@gmail.com] > I don't understand why working code should have to change *twice*. The idea is to make is so that working code only needs to change once, even when supporting multiple Python versions. That one change is to add either an explicit encoding=None (for backwards-compatibility) or an explicit encoding='utf-8' (because that was intended all along). No twice about it, one change. regards, Anders ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BTIXV47LDYPKNIGWNRQSG6LXG4DORS7W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, Feb 10, 2021 at 1:46 AM Anders Munch wrote: >> How about swapping around "locale" and None? Inada Naoki wrote: > > I thought it, but it may not work. Consider about function like this: > > ``` > def read_text(self, encoding=None): > with open(self._filename, encoding=encoding) as f: > return f.read() > ``` > > If `encoding=None` suppresses the warning, functions like this never warned. I don't see why they should be. The author clearly knew about the encoding argument to open, they clearly intended for a None value to be given in some cases, and at the time of writing None meant to use a locale-dependent encoding. > We are not discussing about changing default encoding for now. The section "Prepare to change the default encoding to UTF-8" gave me the impression that this was meant as a stepping stone on the way to doing just that. If that was not the intention, my apologies for the misread. regards, Anders ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2VWZMIBG2VLASF7NCKDEJ5I22PXWI7D7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, 10 Feb 2021 at 14:33, Anders Munch wrote: > > On Tue, 9 Feb 2021 at 16:52, Anders Munch wrote: > >> How about swapping around "locale" and None? That is, make "locale" the > >> new default that emits a warning, and encoding=None emits no warning. > >> That has the advantage that old code can be updated to say encoding=None, > >> and then it will work on both old and new Pythons without warning. > Paul Moore [mailto:p.f.mo...@gmail.com] > > I don't understand why working code should have to change *twice*. > > The idea is to make is so that working code only needs to change once, even > when supporting multiple Python versions. > That one change is to add either an explicit encoding=None (for > backwards-compatibility) or an explicit encoding='utf-8' (because that was > intended all along). No twice about it, one change. But then people who added an explicit utf-8 encoding need to remove the encoding argument again once the default value changes. Your proposal leads to a situation where no-one leaves the encoding argument to default. If we're going to permanently discourage omitting the encoding argument, we should just make it mandatory (a change that I'll argue against, but no-one is currently proposing it, luckily). Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QBX375BJCACY726YHZOXVUOWCG2EH3GI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
Inada Naoki [mailto:songofaca...@gmail.com] wrote: > There are several ways: > * encoding="latin1" -- This is the best. Works perfectly. > * Don't touch -- You don't need to enable EncodingWarning. > [...] I'm replying to Victor's statement that ``encoding="utf8" is backward compatible´´. If you're adding encoding="latin1" to the user program, then you are doing something very different from what Victor proposed. regards, Anders ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IENGLLP67LAOVLVSMJSJX4W6K2ZDTPJ7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
Paul Moore [mailto:p.f.mo...@gmail.com]: wrote: > On Wed, 10 Feb 2021 at 14:33, Anders Munch wrote: >> The idea is to make is so that working code only needs to change once, even >> when supporting multiple Python versions. >> That one change is to add either an explicit encoding=None (for >> backwards-compatibility) or an explicit encoding='utf-8' (because that was >> intended all along). No twice about it, one change. > But then people who added an explicit utf-8 encoding need to remove the > encoding argument again once the default value changes Why would they do that? There's no need to remove anything. Code that doesn't use a default doesn't break because the default changes. regards, Anders ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WBUP2BNLAKHILUM3ZE3A2LVQKAQRXQ7T/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On 2/10/2021 10:29 AM, Paul Moore wrote: On Wed, 10 Feb 2021 at 14:33, Anders Munch wrote: The idea is to make is so that working code only needs to change once, even when supporting multiple Python versions. That one change is to add either an explicit encoding=None (for backwards-compatibility) or an explicit encoding='utf-8' (because that was intended all along). No twice about it, one change. But then people who added an explicit utf-8 encoding need to remove the encoding argument again once the default value changes. Your proposal leads to a situation where no-one leaves the encoding argument to default. If we're going to permanently discourage omitting the encoding argument, we should just make it mandatory (a change that I'll argue against, but no-one is currently proposing it, luckily). Except that all code written after the default has changed (and all python versions without that default are no longer supported) won't need to specify utf-8. And presumably there's more code to be written in the future than already exists. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/EG5UUYGC2R36NPXBDKFTFDKUYSYHP6JR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, 10 Feb 2021 at 16:06, Anders Munch wrote: > > Paul Moore [mailto:p.f.mo...@gmail.com]: wrote: > > On Wed, 10 Feb 2021 at 14:33, Anders Munch wrote: > >> The idea is to make is so that working code only needs to change once, > >> even when supporting multiple Python versions. > >> That one change is to add either an explicit encoding=None (for > >> backwards-compatibility) or an explicit encoding='utf-8' (because that was > >> intended all along). No twice about it, one change. > > > But then people who added an explicit utf-8 encoding need to remove the > > encoding argument again once the default value changes > > Why would they do that? There's no need to remove anything. Code that > doesn't use a default doesn't break because the default changes. Because I'm against a proposal that forces *everyone* to explicitly specify the value... Your argument implies that removing the default altogether would be fine as well. Paul ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GA3EICY5IN7S7VVH24CX3SPAENKCAMFW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Wed, Feb 10, 2021 at 11:58 PM Anders Munch wrote: > > On Wed, Feb 10, 2021 at 1:46 AM Anders Munch wrote: > >> How about swapping around "locale" and None? > Inada Naoki wrote: > > > > I thought it, but it may not work. Consider about function like this: > > > > ``` > > def read_text(self, encoding=None): > > with open(self._filename, encoding=encoding) as f: > > return f.read() > > ``` > > > > If `encoding=None` suppresses the warning, functions like this never warned. > > I don't see why they should be. The author clearly knew about the encoding > argument to open, they clearly intended for a None value to be given in some > cases, and at the time of writing None meant to use a locale-dependent > encoding. > It is not clear. The author may just want to "use the default encoding same to open()". If so, the caller of the function should be warned. To warn caller, this function can use `encoding=io.text_encoding(encoding)` as described in the PEP. > > We are not discussing about changing default encoding for now. > > The section "Prepare to change the default encoding to UTF-8" gave me the > impression that this was meant as a stepping stone on the way to doing just > that. If that was not the intention, my apologies for the misread. > This *can* be stepping stone. But it is not a frist goal. This PEP doesn't discourange omitting encoding option anytime soon when user really need to use locale encoding. Default encoding is used for: a. Really need to use locale specific encoding b. UTF-8 (bug. not work on Windows) c. ASCII (not a bug, but slow on Windows) I assume most usages are (b) and (c). This PEP can reduce them soon. If we decided to change the default encoding in the future, we need to warn omitting encoding option. Reducing (b) and (c) will reduce the total warning shown in the future. This is what "Prepare" means. Additionally, `encoding="locale"` will be backward/forward compatible way to use locale-specific encoding when we decided to change the default encoding. So this PEP can be a very important stepping stone. On the other hand, it is not a problem that we can not use `encoding="locale"` in backward-compatible code *for now*. Python 3.9 become EOL in 2025. We won't emit warning for the default encoding until then. People can use `encoding="locale"` after they drop Python 3.9 support. No problem. Regards, -- Inada Naoki ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DBDI5FEJCF2IOTSAS7VELO27MNEQMK2Z/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: It is really necessary to check that a object has a Py_TPFLAGS_HEAPTYPE flag?
The last time I noticed this question (probably around python 2.4?), it was considered a deliberate decision. There are languages with more open classes, such as (IIRC) Ruby and Smalltalk, but Python chose to remain somewhat closed, because monkey-patching is undesirable, and can be a problem for security audits. Whether it would really open additional attack vectors or crash possibilities ... wasn't judged worth the time to analyze or the risk of being wrong. Maybe those tradeoffs have changed, but if you don't get any other answers, that (and the bias for status quo) is the likely explanation. -jJ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/H3TLXXUEFZMKD4A7RBGUSILSQ6TSMCOI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
I just reread PEP 597, then re-reread the Rationale. The PEP helps when the locale is ASCII or C, but that isn't enforced in actual files. I am confident that this is a frequent problem for packages downloaded from mostly-English sites, including many software repositories. It does not seem to be a win when the locale is something incompatible with utf-8, such as Latin-1, or whatever is still common in Japan. The surrogate-escape mechanism allows a proper round-trip, but python itself will stop processing the characters correctly. For interactive use, when talking to another program (such as a terminal) instead of an already existing file, the backwards compatibility problem seems worse. Changing the default to utf-8 (after a deprecation period showing how to make locale an explicit default) may be reasonable, but claiming that it is backwards compatible ... I didn't get that impression from the PEP. -jJ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RA5SLRB4M7IDLVZKQ3NWVACBLHII2BTR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 597: Add optional EncodingWarning
On Tue, Feb 9, 2021 at 9:51 PM Paul Moore wrote: > * Realistically, I'd be surprised if developers actually use such a > tool. If they were likely to do so, they could probably just as easily > locate all the uses of open() in their code, and check that way. So > I'm not sure this proposal is actually worth it, even if the end > result would be very beneficial. That's ok, they are many Python features which are only used by a minority of users. For me it's similar to the Python Development Mode: https://docs.python.org/dev/library/devmode.html Most users and developers will never use it, but for developers who care, it's a useful tool (it ease the discovery of issues). Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QRYY3M2RXQ7W33DO4TRJBNXWTV6N6BQE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 647 (type guards) -- final call for comments
(1) Is it really a TypeGuard, or more of a TypeAssertion?
(2) Does this push too hard on "annotations only have one meaning"? If it has
to imported from typing, then probably not, but I think that is worth adding to
the PEP.
(3) Why can't the falsey case of an Optional String narrow to a set of
literals {"", None} Are you worried that a subclass of str might have its own
empty string, or just that you don't want to promise this? As written, it
sounds like such a narrowing is forbidden.
-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/XRE57ARQFY7IR7GGZH5EKQZMYD3NM3OY/
Code of Conduct: http://python.org/psf/codeofconduct/
