[issue37186] Everyone uses GIL wrong! = DEADLOCK
New submission from Roffild : Everyone uses GIL wrong! = DEADLOCK I used sub-interpreters in embedded Python: https://github.com/Roffild/RoffildLibrary/blob/35ef39fafc164d260396b39b28ff897d44cf0adb/Libraries/Roffild/PythonDLL/private.h#L44 https://github.com/Roffild/RoffildLibrary/blob/35ef39fafc164d260396b39b28ff897d44cf0adb/Libraries/Roffild/PythonDLL/mql_class.c#L142 PyEval_AcquireThread(__interp->interp); ... PyGILState_Ensure() = DEADLOCK ... PyEval_ReleaseThread(__interp->interp); A deadlock happens in the line: https://github.com/python/cpython/blob/7114c6504a60365b8b0cd718da0ec8a737599fb9/Python/pystate.c#L1313 Of course in the help there is the note: Note that the PyGILState_() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_() API is unsupported. But functions PyGILState_() are used in third-party libraries. Most often, these functions are used without checking that GIL is already locked. Often, these functions are added to the code for reinsurance only and this can affect performance. Numpy: https://github.com/numpy/numpy/blob/2d4975e75c210202293b894bf98faf12f4697a31/numpy/core/include/numpy/ndarraytypes.h#L987 https://github.com/numpy/numpy/search?q=NPY_ALLOW_C_API&unscoped_q=NPY_ALLOW_C_API Pytorch: https://github.com/pytorch/pytorch/blob/0a3fb45d3d2cfacbd0469bbdba0e6cb1a2cd1bbe/torch/csrc/utils/auto_gil.h#L9 https://github.com/pytorch/pytorch/search?q=AutoGIL&unscoped_q=AutoGIL Pybind11 developers have already fixed this problem: https://github.com/pybind/pybind11/blob/97784dad3e518ccb415d5db57ff9b933495d9024/include/pybind11/pybind11.h#L1846 It is necessary to change the code of PyGILState_() functions to support sub-interpreters. Or add to https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock warning: Some Python libraries cannot be used in a sub-interpreter due to the likelihood of deadlock. For me, this is a critical vulnerability! There is another problem: Calling PyEval_AcquireThread() again results in a deadlock. This can be controlled in your code, but not in a third-party library. -- components: Interpreter Core messages: 344891 nosy: Roffild priority: normal severity: normal status: open title: Everyone uses GIL wrong! = DEADLOCK type: crash versions: Python 3.7 ___ Python tracker <https://bugs.python.org/issue37186> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35154] subprocess.list2cmdline() does not allow running some commands.
New submission from Roffild : This issue has already been discussed in #23862 Have to use this hack: import subprocess def list2cmdlineHack(seq): return " ".join(seq) subprocess.list2cmdline = list2cmdlineHack There must be an argument in subprocess.run() to disable subprocess.list2cmdline(). Let the programmer set the start line of the command himself. -- components: IO, Library (Lib), Windows messages: 329207 nosy: Roffild, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: subprocess.list2cmdline() does not allow running some commands. type: behavior versions: Python 3.6, Python 3.7 ___ Python tracker <https://bugs.python.org/issue35154> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35154] subprocess.list2cmdline() does not allow running some commands.
Roffild added the comment:
Yes, the string with spaces must be enclosed in double quotes. But when a
programmer calls CreateProcess(), he himself puts double quotes or calls
CommandLineToArgvW().
subprocess.list2cmdline() is always called and can spoil the launch string.
I propose to add an additional argument list2cmdline=False in subprocess.run()
to disable this function.
subprocess.run(['"program path"', '/param="space space"'], list2cmdline=False)
subprocess.run('"program path" /param="space space"', list2cmdline=False)
--
___
Python tracker
<https://bugs.python.org/issue35154>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35154] subprocess.list2cmdline() does not allow running some commands.
Roffild added the comment: Yes, my mistake. 3.6/Lib/subprocess.py:970 -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue35154> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35223] Pathlib incorrectly merges strings.
New submission from Roffild :
Code:
import os
print(os.path.join("C:/123\\345", "\\", "folder///filename.bin"))
import pathlib
print(pathlib.PureWindowsPath("C:/123\\345", "\\", "folder///filename.bin"))
Result:
C:\folder///filename.bin
C:\folder\filename.bin
Expected result for Windows:
C:\123\345\folder\filename.bin
The number of slashes should be controlled by the library. Replacing / on \
should also depend on the OS.
--
components: IO, Library (Lib), Windows
messages: 329776
nosy: Roffild, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Pathlib incorrectly merges strings.
type: behavior
versions: Python 3.6
___
Python tracker
<https://bugs.python.org/issue35223>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35223] Pathlib incorrectly merges strings.
Roffild added the comment:
It is necessary to assemble a single path from several lines depending on the
OS.
It is logical to expect behavior in Java.
https://docs.oracle.com/javase/8/docs/api/java/nio/file/Paths.html
Converts a path string, or a sequence of strings that when joined form a path
string, to a Path. If more does not specify any elements then the value of the
first parameter is the path string to convert. If more specifies one or more
elements then each non-empty string, including first, is considered to be a
sequence of name elements (see Path) and is joined to form a path string. The
details as to how the Strings are joined is provider specific but typically
they will be joined using the name-separator as the separator. For example, if
the name separator is "/" and getPath("/foo","bar","gus") is invoked, then the
path string "/foo/bar/gus" is converted to a Path. A Path representing an empty
path is returned if first is the empty string and more does not contain any
non-empty strings.
My temporary fix is something like this:
print("\\".join(["C:/123\\345", "\\", "folder///filename.bin"]).replace("/",
"\\").replace("", "\\").replace("", "\\"))
--
___
Python tracker
<https://bugs.python.org/issue35223>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
