Eryk Sun added the comment:
The issue tracker is not the right forum for questions about Python development
or CPython internals. You can ask for help on the python-list mailing list.
--
nosy: +eryksun
resolution: -> not a bug
stage: -> resolved
status: open -&g
Eryk Sun added the comment:
> Maybe it's time to remove Lib/encodings/cp65001.py and make it an
> alias to utf_8 codec
Note that Unicode console support already assumes they're equivalent. For
stdin, we read UTF-16LE via ReadConsoleW and encode bytes to the UTF-8
Bu
Eryk Sun added the comment:
> libc = ctypes.cdll.msvcrt
That's the private CRT of Windows, not the Universal CRT for applications. In a
release build (python.exe), use ctypes.CDLL('ucrtbase', use_errno=True). In a
debug build (python_d.exe), use ctypes.CDLL('ucrtbase
Eryk Sun added the comment:
> FYI, I expect cp65001 will be used more widely in near future,
[...]
> It seems use `SetConsoleOutputCP(65001)` and `SetConsoleCP(65001)`.
Unless PYTHONLEGACYWINDOWSSTDIO is defined, Python 3.6+ doesn't use the
console's codepage-based interface
Eryk Sun added the comment:
Windows resolve() is based on ntpath._getfinalpathname, which requires that the
file exists and is accessible. resolve() defaults to non-strict behavior, which
tries to resolve as much as possible by traversing the path in reverse and
trying _getfinalpathname
Eryk Sun added the comment:
> I dislike lying in the locale module. This change is basically useless
> with my PR 13230.
Yes, functionally it's no different than using 'cp65001' as an alias. That
said, the CRT special cases 65001 as "utf8":
>&
Eryk Sun added the comment:
This is due to an oversight in _CallPythonObject in
Modules/_ctypes/callbacks.c. The setfunc protocol is to return None if there's
no object to keep alive. This isn't applicable to py_object (i.e. O_set in
Modules/_ctypes/cfield.c). So the prob
Eryk Sun added the comment:
> For tests, you should be able to create your own REG_MULTI_SZ key with
> zero-length strings and read it back. If the winreg module won't let
> you do it, you may need ctypes (but that's okay in Windows-only
> tests).
winreg.SetValueEx ca
Eryk Sun added the comment:
This was my fault for recommending PyErr_SetFromWindowsErrWithUnicodeFilename
without checking the header for deprecation.
PyErr_SetFromWindowsErrWithUnicodeFilename is an internal function (added for
PEP 277, circa 2.4), which was never documented in the C API
Eryk Sun added the comment:
This should be fixed. That said, we have to use a unicode string for a long
path anyway. Prior to Windows 8, the conversion from ANSI to Unicode in the
system runtime library uses a static MAX_PATH buffer, so the ANSI API is
inherently limited to MAX_PATH. You&#x
Change by Eryk Sun :
--
stage: -> patch review
type: -> behavior
___
Python tracker
<https://bugs.python.org/issue32539>
___
___
Python-bugs-list
Eryk Sun added the comment:
> on Windows, the rpath mechanism doesn't exist
It seems we can locate a dependent assembly up to two directories up from a DLL
using a relative path. According to MSDN [1], this is supported in Windows 7+.
3.7 no longer supports Vista, so this can potent
Eryk Sun added the comment:
This is the high-level shutil module, so why not try to use the resolved parent
directory? For example:
def disk_usage(path):
try:
total, free = nt._getdiskusage(path)
except NotADirectoryError:
path = os.path.dirname(nt
New submission from Eryk Sun :
In issue 26330 it was noted that nt._getdiskusage doesn't accept bytes paths.
The same applies to nt._getvolumepathname and nt._getfinalpathname. Callers of
these functions shouldn't have to manually implement PEP 529 UTF-8 support to
handle input
New submission from Eryk Sun :
Issue 26330 was resolved by documenting that shutil.disk_usage requires a
directory. However, the shutil module is in a position to harmonize
cross-platform behavior in ways that aren't normally possible or recommended in
the low-level os module. To tha
New submission from Eryk Sun :
I've occasionally seen requests to find the path of the LNK shortcut that was
used to run a script -- for whatever reason. This can be reliably supported in
most cases by calling WinAPI GetStartupInfo. If the flag STARTF_TITLEISLINKNAME
is set, then the lp
Eryk Sun added the comment:
`PATHEXT` is irrelevant here, so that should be removed. As to the py launcher,
it's optional and not always installed for all users, though that's the
default. I'd prefer `python` that's found in PATH as the first example. Then
add another e
Eryk Sun added the comment:
On Windows it's the directory that contains the zip file or directory with
__main__.py, not the current directory. This seems normal to me. The directory
or zip file is effectively executing as a script. I can understand wanting more
isolated behavior in this
Eryk Sun added the comment:
PyWin32's win32api.RegQueryValueEx and win32api.RegEnumValue also use the same
documented interpretation of REG_MULTI_SZ [1] (i.e. a "sequence of
null-terminated strings, terminated by an empty string").
[1]: https://msdn.microsoft.com/en-us/l
Eryk Sun added the comment:
CPython pushes a frame on the thread's stack for every Python frame. By default
in Windows Python the thread's stack is allowed to grow up to about 2 MB. On
Linux the default limit is 8 MB, but a higher limit can be set via `ulimit -s`.
After the sta
Eryk Sun added the comment:
Currently countStrings and fixupMultiSZ halt the outer loop as soon as they hit
an empty string. Basically, we'd be getting rid of that check. Then we have to
prevent including an empty string for the final NUL in the normal case. We can
do this by decreme
New submission from Eryk Sun :
os.getlogin() is supposed to return the name of the user logged in on the
"controlling terminal" of the process. Windows doesn't have POSIX terminals;
however, every process does belong to a terminal/desktop session that can be
connected either
Change by Eryk Sun :
--
resolution: -> duplicate
stage: -> resolved
status: open -> closed
superseder: -> support bytes paths in nt _getdiskusage, _getvolumepathname,
and _getfinalpathname
___
Python tracker
<https://bugs.python
Eryk Sun added the comment:
PyUnicode_AsWideCharString was updated to raise ValueError for embedded nulls
if the `size` output parameter is NULL. Z_set in cfield.c should be updated to
get the size, which can be ignored here. For example:
Py_ssize_t size;
buffer
Eryk Sun added the comment:
If a PathLike args value is supported in Windows, then it must be processed
through list2cmdline, in case it needs to be quoted. Also, the preferred usage
is to pass args as a list when shell is false. This common case shouldn't be
penalized as a TypeError.
Eryk Sun added the comment:
Is it just for this particular svn.exe application, or is it for any
application (e.g. tasklist.exe)?
The symptoms sound like svn.exe is spawning a new instance of itself with the
`CREATE_NEW_CONSOLE` creation flag, which allocates a new console and replaces
the
Eryk Sun added the comment:
Setting CREATE_NEW_CONSOLE in Python isn't the same since Popen explicitly sets
the standard handles via STARTUPINFO.
I installed TortoiseSVN-1.9.7.27907-x64-svn-1.9.7.msi in Windows 10 release
1709, but I can't reproduce your issue. Does it work correct
Eryk Sun added the comment:
zz2_err.py uses LF line endings instead of CRLF line endings. This is
incompatible with text mode I/O in Microsoft C, which leads to undefined
behavior. See issue 20844 and issue 27797.
--
nosy: +eryksun
resolution: -> duplicate
stage: -> resolved
Eryk Sun added the comment:
Work on related issues extends back 9 years to issue 5115. Also see issue 26534.
--
nosy: +eryksun
___
Python tracker
<https://bugs.python.org/issue32
New submission from Eryk Sun :
File descriptors in Windows are implemented by the C runtime library's low I/O
layer. The CRT maps native File handles to Unix-style file descriptors.
Additionally, in order to support inheritance for spawn/exec, the CRT passes
inheritable FDs in a res
Eryk Sun added the comment:
Note that the CRT checks at startup whether an inherited FD is valid by calling
GetFileType. If the handle is invalid or not a File, then the FD effectively is
not inherited. This doesn't completely avoid the problem, since there's still a
chance the
Eryk Sun added the comment:
In Windows the CRT file descriptor is actually still inheritable. This only
makes the underlying OS handle non-inheritable. I don't think there's a way to
make an existing FD non-inheritable using public CRT functions. See issue 32865.
--
nosy
Eryk Sun added the comment:
> The first is a bug in os.pipe (creation of an inheritable descriptor
> with non-inheritable underlying handle). This can be fixed by using
> _open_osfhandle() correctly.
This is the only issue to be addressed here, and it's an easy fix. The problem
New submission from Eryk Sun :
When reviewing issue 32904 I noticed that os.getcwdb still calls the CRT
_getcwd function. Apparently this was overlooked when implementing PEP 529. For
example:
>>> os.getcwd()
'C:\\Temp\\Lang\\αβγδ'
>>> os.getcwdb()
Eryk Sun added the comment:
>> As os.symlink requires administrative privileges on most versions
>> of Windows
>
> The current implementation requires SeCreateSymbolicLinkPrivilege on
> ALL versions of Windows because users must pass an additional flag to
> CreateSy
Eryk Sun added the comment:
Yes, it makes sense to call GetFileAttributes as the fast path, and then fall
back on a slow path (i.e. create, query, close) for a directory reparse point.
With GetFileInformationByHandleEx, the equivalent query is FileBasicInfo, which
is available starting with
Eryk Sun added the comment:
Thanks for the quick feedback and pushing back on this. We just have a broken
mount point, unlike in Unix where it's a regular directory. so you're right to
just query the basic info for all directory reparse points, without
special-casing mount points
Eryk Sun added the comment:
The inconsistent use of VOLUME_NAME_NT and VOLUME_NAME_DOS was addressed
incidentally in issue 29734, but that issue is primarily about the handle
leaked when GetFinalPathNameByHandleW fails. That said, you've cleanly
addressed the handle leak in your PR al
Eryk Sun added the comment:
FYI, here's a sampling of successful calls that modify the last error value.
Most Create- functions intentionally set the last error to 0 on success, such
as CreateFile, CreateFileMapping, CreateSymbolicLink, CreateJobObject,
CreateEvent, Create
Eryk Sun added the comment:
> pathlib currently expects DOS paths only: it will strip '\\?\'
> prefix in resolve()
pathlib's unqualified conversion from the extended form to classic DOS form is
a bug. The resolved path may be invalid as a classic DOS path. It may be to
Eryk Sun added the comment:
Python 2.7 is all but set in stone. Changes to its behavior have to correct
serious bugs, not work around limits in an OS. You can do that yourself. For
example, use an extended local-device path, i.e. a path that's prefixed by
u"?\\". This pa
Eryk Sun added the comment:
> If you use os.listdir() on the networked folder, the log file
> will come up.
Querying a file's parent directory (e.g. via os.scandir in Python 3) can
provide a basic stat (i.e. attributes, reparse tag, size, and timestamps) when
opening the file dir
Eryk Sun added the comment:
> the way `python -m pip` searches for the module to execute is much
> closer to the way Windows searches for a command like `pip` (i.e.
> current directory first)
That's classic Windows behavior. However, search paths for CreateProcess and
New submission from Eryk Sun :
shutil.chown is defined in Windows even though it's only written for Unix and
only documented as available in Unix. Defining it should be skipped on Windows.
Possibly in 3.8 shutil.chown could be implemented on Windows by calling a new
os.set_owner function
Eryk Sun added the comment:
Oops, the SID for BUILTIN\Administrators is S-1-5-32-544. I left out the
authority ID (5).
--
___
Python tracker
<https://bugs.python.org/issue33
Eryk Sun added the comment:
> if not available fallback on
> GetActiveProcessorCount(ALL_PROCESSOR_GROUPS)
The fallback for older versions of Windows is dwNumberOfProcessors from
GetSystemInfo. This can be removed from 3.7 and 3.8, which no longer support
Windows versions prior to Win
Eryk Sun added the comment:
This is not an uncommon problem. If there's one or more existing references to
a file or empty directory that were opened with shared delete access, then a
delete operation will succeed, but the file or directory will not be unlinked
from the parent directory
Eryk Sun added the comment:
Field names define CField descriptor attributes on the class. Attribute names
should be strings, not bytes. There's no syntactically clean way to use a bytes
name. Consider the example of a generic property on a class:
>>> T = type('T&#x
Change by Eryk Sun :
--
resolution: not a bug -> rejected
___
Python tracker
<https://bugs.python.org/issue33242>
___
___
Python-bugs-list mailing list
Un
Eryk Sun added the comment:
If you're automatically wrapping a C source file and don't know the source
encoding, you could naively decode it as Latin-1. You're still faced with the
problem of characters that Python doesn't allow in identifiers. For example,
gcc allows
Eryk Sun added the comment:
A sub-millisecond wait is fairly quick, but it depends on the machine speed. I
should have included a counter. Try the following. It's not reproducing the
problem if num_retries doesn't get incremented.
import os
import time
ERROR_DIR_NOT_E
Eryk Sun added the comment:
Unless you're willing to step through hoops using ctypes or PyWin32, sending
Ctrl+Break requires already being attached to the same console as the target.
Note that the target isn't necessarily a single process that's attached to the
console. Se
Eryk Sun added the comment:
Your case probably isn't due to a anti-malware filesystem filter. Explorer
keeps handles open to directories to get updates via ReadDirectoryChangesExW.
It opens watched directories with shared delete access, so deleting the child
succeeds. But as discussed
Eryk Sun added the comment:
> The sortedness of glob.glob's output is platform-dependent.
It's typically file-system dependent (e.g. NTFS, FAT, ISO9660, UDF) -- at least
on Windows. NTFS and ISO9660 store directories in sorted order based on the
filename (Unicode or ASCII
Eryk Sun added the comment:
As I said, some file systems such as NTFS and ISO 9660 (or Joliet) store
directories in lexicographically sorted order. NTFS does this using a b-tree
and case-insensitive comparison, which helps the driver efficiently implement
filtering a directory listing using
Eryk Sun added the comment:
For POSIX systems, try the following test function several times. For the bug
to manifest, Thread._wait_for_tstate_lock has to be interrupted in between
acquiring and releasing the sentinel lock. Maybe it could use a reentrant lock
in order to avoid this problem
Eryk Sun added the comment:
There isn't a problem in most cases, since functions that take a callback
usually call it before returning. If the library does keep a long-lived
reference to a callback (e.g. a console control handler in Windows), there are
ways to support this, depending o
Eryk Sun added the comment:
FAT inserts a new file entry in a directory at the first available position.
(If it's a long filename, this could be up to 21 contiguous dirents for a
combined long/short dirent set.) This means a directory listing is usually in
the same order that files
Eryk Sun added the comment:
There's no simple workaround for this behavior. All we can reasonably do is
document that running a batch script directly has the same security risks as
using shell=True.
CMD doesn't support a file argument. It only supports running a /c or /k
command,
Eryk Sun added the comment:
The exec functions provided by the Windows C runtime really are practically
useless, due to creating an orphaned process, disrupting synchronous operation,
and returning the wrong status code. It might be more useful for Python 3.7.x
to implement an internal
Change by Eryk Sun :
--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
___
Python tracker
<https://bugs.python.org/issue19124>
___
___
Eryk Sun added the comment:
The "(I)" flag in an icacls entry means it's inherited from the parent
directory. The installer doesn't override these inherited permissions.
Currently, it's your responsibility to do this if you install to a custom
directory such as C:\
Eryk Sun added the comment:
This is explained in the docs:
Join one or more path components intelligently If a
component is an absolute path, all previous components are
thrown away and joining continues from the absolute path
component. On Windows, the drive letter is
Eryk Sun added the comment:
mktime() is the inverse of localtime(). With the given format string,
strptime() sets tm_isdst to -1, for which mktime will use the system's timezone
information to determine whether or not it's daylight saving time. You need to
verify that the time zon
Eryk Sun added the comment:
This depends on the way the platform mktime implements folding when the clock
is advanced for daylight saving time. It appears the timezone on your systems
is set to US/Canada Central Time, for which on March 9th the clocks were
advanced forward at 02:00:00 [1
Eryk Sun added the comment:
Additionally, the FileIO documentation states the following:
The read() (when called with a positive argument), readinto() and
write() methods on this class will only make one system call.
The Linux man page for write() in turn states this:
On Linux
Eryk Sun added the comment:
AFAIK, the "Advanced system settings" dialog has always required administrator
access. To modify the environment for just the current user, in the control
panel there's "User Accounts" => "Change my environment varia
Eryk Sun added the comment:
This is standard Windows API behavior for the final path component. A single
dot component means the current directory. Two dots means the parent directory.
More than two dots and/or trailing spaces, gets reduced to a single dot,
meaning the current directory. For
Eryk Sun added the comment:
Setting the exit code to the negative of a C signal value isn't generally
meaningful in Windows. It seems multiprocessing doesn't have a significant use
for this, other than getting a formatted exit code in the repr via its
_exitcode_to_name dict. F
Change by Eryk Sun :
--
stage: -> needs patch
versions: +Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue31863>
___
___
Python-bugs-lis
Eryk Sun added the comment:
A C/C++ program returns EXIT_FAILURE for a generic failure. Microsoft defines
this macro value as 1. Most tools that a user might use to forcibly terminate a
process don't allow specifying the reason; they just use the generic value of
1. This includes
Change by Eryk Sun :
--
versions: +Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue31884>
___
___
Python-bugs-list mailing list
Unsub
Change by Eryk Sun :
--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
___
Python tracker
<https://bugs.python.org/issue31884>
___
___
Eryk Sun added the comment:
If a multiprocessing Process gets terminated by any means other than its
terminate() method, it won't get this special TERMINATE (0x1) exit code
that allows the object to pretend the exit status is POSIX -SIGTERM. In
general, the exit code will be 1
Eryk Sun added the comment:
The error is in _io__WindowsConsoleIO_write_impl. If it's passed a length 0
buffer, it still tries to decode it via MultiByteToWideChar, which fails as
documented. As Serhiy says, it can simply return Python int(0) in the
zero-length case.
--
Eryk Sun added the comment:
The values are correct for the given type codes, which should be the same as
the corresponding type codes for the array and struct modules. Except the array
module doesn't support the "c" type.
However, assigning b's' to an index of a &
Change by Eryk Sun :
--
components: +Library (Lib) -asyncio
type: -> behavior
___
Python tracker
<https://bugs.python.org/issue32003>
___
___
Python-bugs-lis
Change by Eryk Sun :
--
components: +Windows
nosy: +paul.moore, tim.golden, zach.ware
stage: -> test needed
___
Python tracker
<https://bugs.python.org/issu
Eryk Sun added the comment:
We need a test that reproduces this problem on a vanilla installation of Python
3.5. Include the system locale context as well, i.e. the ANSI codepage, OEM
codepage, and active console output codepage. A reliable test will help
determine whether this problem also
Eryk Sun added the comment:
I was able to reproduce this problem in 3.6 in Windows 10 (1709), but only for
code paths that call WriteFile, i.e. os.write and legacy standard I/O mode.
Writing to the console will block if there's an active text selection. The
operation completes once the
Eryk Sun added the comment:
ipconfig uses (or defaults to) OEM encoded output when writing to a pipe or
file. On the other hand, Python's TextIOWrapper defaults to ANSI (i.e. 'mbcs').
In 3.6+, uuid._ipconfig_getnode could be rewritten to call subprocess.Popen
with the new
Eryk Sun added the comment:
run_file encodes the file path via PyUnicode_EncodeFSDefault, which encodes as
UTF-8 in Windows, starting with 3.6. PyRun_SimpleFileExFlags subsequently tries
to open this encoded path via _Py_fopen, which calls fopen. The CRT expects an
ANSI encoded path, so only
Eryk Sun added the comment:
Workarounds: (1) force 3.6 to use the legacy ANSI filesystem encoding by
setting the environment variable PYTHONLEGACYWINDOWSFSENCODING. (2) Use 8.3 DOS
names, if creating them is enabled on the volume. You can check their value in
CMD via `dir /x`. (3) Create
Eryk Sun added the comment:
The CMD shell decodes batch scripts using the attached console's output
codepage, which defaults to OEM. OTOH, venv writes the replacement values for
the template activate.bat as UTF-8 (codepage 65001), which is correct and
should not be downgraded t
Eryk Sun added the comment:
> the UNC path is not really useful anywhere in the Python
> standard library
UNC paths can be used almost anywhere in the file API. What specifically isn't
working?
> there’s no way to turn the it (back) into network drive once you
> call r
Eryk Sun added the comment:
If none of the components of a relative path exist, then splitting off the head
returns an empty string in the second-to-last pass through the while loop. In
the last pass, _getfinalpathname("") raises FileNotFoundError, and ultimately
resolve() ends up
Eryk Sun added the comment:
resolve() has additional problems, which possibly could be addressed all at
once because it's a small method and the problems are closely related.
For an empty path it returns os.getcwd(). I don't think this case is possible
in the normal way a
Eryk Sun added the comment:
Here's a way to trigger this error that's unrelated to the PATH environment
variable:
>>> subprocess.call('python', executable=r'C:\Program
Files\Python36\.\python.exe')
Fatal Python error: Py_Initializ
Eryk Sun added the comment:
The 3.5 branch only gets security fixes at this point. Consider upgrading to
3.6.
That said, this shouldn't be a problem with shutil.get_terminal_size() in 3.5.
It handles AttributeError, ValueError, and OSError by returning the `fallback`
size. And it'
Eryk Sun added the comment:
For extra measure, you may want to normalize `prefix` prior to calculating its
length n in gotlandmark(). Then it would be reliable to truncate it via
`prefix[n] = '\0'` after joining with `landmark`. Or at least add a comment
there or in the callin
Eryk Sun added the comment:
In Python/pythonrun.c, PyRun_SimpleFileExFlags() reopens the PYC file in binary
mode, passes it to run_pyc_file(), and only closes it after executing the
script. The file should instead be closed in run_pyc_file(), before calling
PyEval_EvalCode
Eryk Sun added the comment:
The thread handle that CreateProcess returns gets immediately closed in
Popen._execute_child, so I can't see how this is due to subprocess. Please
check to make sure these thread handles aren't for threads in your own process.
Set the lower pane
Eryk Sun added the comment:
The 2nd example with subprocess.run() creates two threads in the Python
process, since you're redirecting both stdout and stderr to pipes and run()
calls communicate(). The first example with subprocess.Popen() shouldn't create
any threads. In either cas
Eryk Sun added the comment:
This behavior is inherited from BaseException:
https://docs.python.org/3/library/exceptions.html#BaseException
>>> str(BaseException())
''
>>> str(BaseException('spam'))
'spam'
>>&g
Eryk Sun added the comment:
It has to be a ValueError since the error is an invalid parameter at the Python
level. Similarly, in 3.6+ when using bytes paths in Windows, for which Python
uses UTF-8 as the file-system encoding, os.path.exists() may raise
UnicodeDecodeError:
>>
Eryk Sun added the comment:
> To get the correct output, cmd has a "/u" switch (this switch has
> probably existed forever - at least since Windows NT 4.0, by my
> internet search). The output can then be decoded using
> encoding='utf-16-le', like any native
Eryk Sun added the comment:
> By default, the output of cmd is encoded with the "active"
> codepage. In Python 3.6, you can decode this using
> encoding='oem'.
FYI, the actual encoding is not necessarily "oem".
The console codepage may have b
Eryk Sun added the comment:
>> It has to be a ValueError since the error is an invalid
>> parameter at the Python level.
>
> How does the first follow from the second?
I only meant that, as an honest error, it has to be ValueError. I didn't think
about raising a fak
Eryk Sun added the comment:
With a stock Windows system, `subprocess.check_output(['echo', "'hello'"])`
fails because there is no "echo.exe". That's a 3rd party program that you
installed. `subprocess.check_output("echo 'hello'&
Eryk Sun added the comment:
> I don't think is cross-platform, because I'm still on Windows but
> in different shells.
MSYS/MINGW64 and Cygwin are POSIX runtime environments implemented in DLLs, but
they're layered over Windows and can thus support Windows paths s
1501 - 1600 of 2119 matches
Mail list logo
