[issue43395] os.path states that bytes can't represent all MBCS paths under Windows
New submission from Eric L. : The os.path documentation at https://docs.python.org/3/library/os.path.html states that: > Vice versa, using bytes objects cannot represent all file names on Windows > (in the standard mbcs encoding), hence Windows applications should use string > objects to access all files. This doesn't sound right and is at least misleading because anything can be represented as bytes, as everything (in a computer) is bytes at the end of the day, unless mbcs is really using something like half-bytes, which I couldn't find any sign of (skimming through the documentation, Microsoft seems to interpret it as DBCS, one or two bytes). I could imagine that the meaning is that some bytes combinations can't be used as path under Windows, but I just don't know, and that wouldn't be a valid reason to not use bytes under Windows (IMHO). -- assignee: docs@python components: Documentation messages: 388077 nosy: docs@python, ericzolf priority: normal severity: normal status: open title: os.path states that bytes can't represent all MBCS paths under Windows type: enhancement versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43395] os.path states that bytes can't represent all MBCS paths under Windows
Eric L. added the comment: Very confusing but very interesting. I'm trying to follow as I'm the main maintainer of the rdiff-backup software, which goes cross-platforms, so these small differences might become important. Now, looking into the docs, following your explanations, I noticed that https://docs.python.org/3/library/os.html#os.fsencode and https://docs.python.org/3/library/os.html#os.fsdecode state that the 'strict' error handler is used under Windows instead of the stated 'surrogatepass'. Again an issue with the documentation? Also, the 2nd paragraph of https://docs.python.org/3.8/library/os.html#file-names-command-line-arguments-and-environment-variables speaks only of surrogateescape and doesn't make the difference between POSIX and Windows. Very interesting but very confusing... -- ___ Python tracker <https://bugs.python.org/issue43395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40701] tempfile mixes str and bytes in an inconsistent manner
New submission from Eric L. :
tempfile fails on mixed str and bytes when setting tempfile.tempdir to a
non-existent bytes path but succeeds when set to an existing bytes path.
$ python3.9
Python 3.9.0a6 (default, Apr 28 2020, 00:00:00)
[GCC 10.0.1 20200430 (Red Hat 10.0.1-0.14)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> tempfile.tempdir = b'/doesntexist'
>>> tempfile.TemporaryFile()
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.9/tempfile.py", line 615, in TemporaryFile
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
File "/usr/lib64/python3.9/tempfile.py", line 248, in _mkstemp_inner
file = _os.path.join(dir, pre + name + suf)
File "/usr/lib64/python3.9/posixpath.py", line 90, in join
genericpath._check_arg_types('join', a, *p)
File "/usr/lib64/python3.9/genericpath.py", line 155, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
>>> tempfile.tempdir = b'/tmp'
>>> tempfile.TemporaryFile()
<_io.BufferedRandom name=3>
>>> tempfile.mktemp()
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib64/python3.9/tempfile.py", line 400, in mktemp
file = _os.path.join(dir, prefix + name + suffix)
File "/usr/lib64/python3.9/posixpath.py", line 90, in join
genericpath._check_arg_types('join', a, *p)
File "/usr/lib64/python3.9/genericpath.py", line 155, in _check_arg_types
raise TypeError("Can't mix strings and bytes in path components") from None
TypeError: Can't mix strings and bytes in path components
It seems to me that the case of `tempfile.tempdir` being of type bytes hasn't
been completely considered and is handled inconsistently. My suggestion would
be to manage the paths as bytes if there is no other indication like suffix or
prefix.
--
messages: 369469
nosy: ericzolf
priority: normal
severity: normal
status: open
title: tempfile mixes str and bytes in an inconsistent manner
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue40701>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40701] tempfile mixes str and bytes in an inconsistent manner
Eric L. added the comment: In the meantime, I noticed the following in addition: [ericl@tuxedo ~]$ python3.9 Python 3.9.0a6 (default, Apr 28 2020, 00:00:00) [GCC 10.0.1 20200430 (Red Hat 10.0.1-0.14)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import tempfile >>> tempfile.tempdir = b'/tmp' >>> tempfile.gettempdir() b'/tmp' >>> tempfile.tempdir = '/tmp' >>> tempfile.gettempdirb() b'/tmp' This actually explicitly hurts the interface description which states that tempfile.gettempdir() returns a string. "Encouraged" by this discovery, I've tried to write a patch of tempfile.py addressing the issues discovered. It's my first patch ever of Python so bare with me. The default remains string but if someone _explicitly_ sets tempdir to bytes, it'll become bytes. I've tried all the commands listed previously and it all looks consistent to me. -- keywords: +patch Added file: https://bugs.python.org/file49176/tempfile.py.diff ___ Python tracker <https://bugs.python.org/issue40701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40701] tempfile mixes str and bytes in an inconsistent manner
Eric L. added the comment: Sorry, I uploaded by mistake an early version of the patch. The new one is the one I had actually tested (the old one would fail with mixing bytes and string under certain circumstances, I can't remember any more). -- Added file: https://bugs.python.org/file49185/tempfile_py_2020-05-23.diff ___ Python tracker <https://bugs.python.org/issue40701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40701] tempfile mixes str and bytes in an inconsistent manner
Eric L. added the comment: On 23/05/2020 21:41, Gregory P. Smith wrote: > Could you please turn that into a Github PR? I can, if you don't tell me that I need to setup a full-blown Python development environment to do this. -- ___ Python tracker <https://bugs.python.org/issue40701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40701] tempfile mixes str and bytes in an inconsistent manner
Eric L. added the comment: On 24/05/2020 20:30, Serhiy Storchaka wrote: > Maybe just document that tempdir should be a string? I would definitely prefer to have bytes paths considered as 1st class citizen. -- ___ Python tracker <https://bugs.python.org/issue40701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40701] tempfile mixes str and bytes in an inconsistent manner
Eric L. added the comment: Well, your decision but, as a user of the library, it didn't feel like a new feature just like a bug to be fixed, the main issue being the inconsistent handling of bytes vs. str. -- ___ Python tracker <https://bugs.python.org/issue40701> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40831] Wrong statement that bytes paths are deprecated under Windows regarding MAX_PATH docs
New submission from Eric L. : In chapter 3.1.2. Removing the MAX_PATH Limitation of https://docs.python.org/3/using/windows.html#removing-the-max-path-limitation (also in 3.9 and 3.10), it is written that "(Use of bytes as paths is deprecated on Windows, and this feature is not available when using bytes.)" but my understanding is that since 3.8 bytes paths are _not_ deprecated any more under Windows (and it is good so!), which makes unclear if the feature is still not available with bytes paths. In which case, it should be implemented. I'm sorry I couldn't test if it actually works or not but I'm stuck with Python 3.7 where I guess the statement is still correct, but I hope to see the issue fixed once I can upgrade :-) -- components: Windows messages: 370414 nosy: ericzolf, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Wrong statement that bytes paths are deprecated under Windows regarding MAX_PATH docs type: behavior versions: Python 3.10, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue40831> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40831] Wrong statement that bytes paths are deprecated under Windows regarding MAX_PATH docs
Eric L. added the comment: The question is if only the statement that bytes are deprecated is wrong, but also if the long path feature is supported with bytes or not. As written, I'm a Linux guy, I don't feel like I have the pre-requisites to check this properly. -- ___ Python tracker <https://bugs.python.org/issue40831> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3467] sqlite3 path is hard coded in setup.py
New submission from Eric L. Frederich <[EMAIL PROTECTED]>: In setup.py, the paths that it searches for sqlite in is hard coded in a list sqlite_inc_paths. This should also search any path in either $PATH or $LD_LIBRARY_PATH. This is necessary for non-default installations of sqlite for users without root access, or for those with root access that have multiple versions installed in different locations. -- components: Installation messages: 70411 nosy: ericfrederich severity: normal status: open title: sqlite3 path is hard coded in setup.py type: compile error versions: Python 2.5 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue3467> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue3467] sqlite3 path is hard coded in setup.py
Eric L. Frederich <[EMAIL PROTECTED]> added the comment:
If we put the following one liner right after sqlite_inc_paths is
defined it will add include directories based on the PATH environment
variable.
sqlite_inc_paths.extend([re.sub('/bin[/]?$', '/include', p) for p in
os.environ.get('PATH', '').split(':')])
___
Python tracker <[EMAIL PROTECTED]>
<http://bugs.python.org/issue3467>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
