Gregory P. Smith added the comment:
Basically you want it to call fchdir() instead of chdir() when passed a fd
(integer) instead of a string/Path-like. That makes sense and should be a
reasonably straight forward set of changes to _posixsubprocess.c.
(A way to convert a fd into a Path-like
Gregory P. Smith added the comment:
Examining Lib/zipfile.py code, the existing code makes sense. Python's zipfile
module produces modern zipfiles when writing by setting the utf-8 flag and
storing the filename as utf-8 when it is not ASCII. This is desirable for use
with all norma
Gregory P. Smith added the comment:
Your PR looks good to me.
I agree with not making it easy to _write_ zipfiles with non-standard encoding
used for names.
There is a possibility that someone wants that ability when writing zip files
(not yet clear) in https://bugs.python.org/issue40172
New submission from Gregory P. Smith :
We have a pile of conditionals and extra code in CPython to deal with building
on systems that do not have zlib. The zlib C library has been around forever
at this point and should be present on every system in the world.
zlib is already required on
Change by Gregory P. Smith :
--
keywords: +patch
pull_requests: +30133
pull_request: https://github.com/python/cpython/pull/32043
___
Python tracker
<https://bugs.python.org/issue47
Gregory P. Smith added the comment:
hashlib creator and other maintainer here: I do not think it was a good idea
for us to add blake2 to hashlib the way we did. So blake3 should not be
presumed as a given, at least not done in the same manner.
Background:
While OpenSSL gained _some_ blake2
Gregory P. Smith added the comment:
Bringing this up on discord, others point out that the windows build requires
zlib for convenience when we transitioned from having a vendored copy in our
repo and that smaller "embedded" use cases may not like this if they don't
already ne
Gregory P. Smith added the comment:
correction: our md5/sha1/sha2/sha3 code is not gone yet, but they are simple C
implementations used as a fallback when the provider of optimal versions are
unavailable (openssl for those). That keeps the copies of code in our tree
simple and most people
Gregory P. Smith added the comment:
In the short term we should prefer libb2 linkage when available.
As for deprecation, it'd be useful to research how often the options going away
are used in code in PyPI packages and in Github repos to understand the
deprecation impact.
The
Gregory P. Smith added the comment:
Here's how I created an IPv6-only host:
https://gist.github.com/gpshead/f4f394593674e5f7a58e9424b4dba989
--
___
Python tracker
<https://bugs.python.org/is
Gregory P. Smith added the comment:
Because I don't think blake3 or blake2 _(though we've shipped it already so
there's a challenge in making changes https://bugs.python.org/issue47095)_ are
important enough to be _guaranteed_ present in all builds (our release binaries
wou
New submission from Gregory P. Smith :
Linux kernels provide a CryptoAPI. This is a common place for platform specific
hardware accelerated hash algorithms to be exposed to the user (especially on
SoCs which often have non-standard hardware).
https://www.kernel.org/doc/html/v4.10/crypto
Gregory P. Smith added the comment:
To anyone else who comes along with motivation:
I'm fine with blake3 being in hashlib, but I don't want us to guarantee it by
carrying the implementation of the algorithm in the CPython codebase itself
unless it gains wide industry standard-lik
Gregory P. Smith added the comment:
Performance wise... The SHA series have hardware acceleration on modern CPUs
and SoCs. External libraries such as OpenSSL are in a position to provide
implementations that make use of that. Same with the Linux Kernel CryptoAPI
(https://bugs.python.org
Gregory P. Smith added the comment:
Neat. I've never used the API, just filing a breadcrumb suggesting we see if it
makes sense. Being I/O based, that even takes care of GIL releasing from
Python. :)
--
___
Python tracker
&
Gregory P. Smith added the comment:
Rust based anything comes with a baseline level of Rust code overhead.
https://stackoverflow.com/questions/29008127/why-are-rust-executables-so-huge
That seems expected.
--
___
Python tracker
<ht
Gregory P. Smith added the comment:
Thanks Serhiy!
--
resolution: -> fixed
stage: patch review -> commit review
status: open -> closed
___
Python tracker
<https://bugs.python.or
Gregory P. Smith added the comment:
You missed the key "And certainly more efficient in terms of watt-secs/byte"
part.
--
___
Python tracker
<https://bugs.python.o
Gregory P. Smith added the comment:
Bug filing tip for ty/zonyitoo: Describe the problem in the text when filing
the bug. Include its consequences and how it is observed in practice.
A bug filed just saying "look at this code, this is wrong" does not communicate
a need for anyon
Gregory P. Smith added the comment:
It looks like the length would be short by one in Python before this change,
meaning binding or connecting to a non-anonymous named AF_UNIX socket
potentially loses the last character of the path name intended to be bound?
Depending on if the OS uses the
Change by Gregory P. Smith :
--
assignee: -> gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue44493>
___
___
Python-bugs-list mailing list
Un
New submission from Gregory P. Smith :
https://developer.apple.com/documentation/cryptokit/ in macOS 10.15+
This is a common place for platform specific hardware acceleration to be
exposed to the user (especially on SoCs which often have non-standard hardware
- Like Apples... which is
New submission from Gregory P. Smith :
https://docs.microsoft.com/en-us/windows/win32/seccng/creating-a-hash-with-cng
See if these are worthwhile vs using OpenSSL for the hashlib algorithms it
supports. OS APIs can in theory take better advantage of HW acceleration for
performance. Verify
Gregory P. Smith added the comment:
I only pointed to that API after a brief search without looking at details
(Swift? oops!). If there is one available from C that'd also make sense to
consider.
The only things I expect, relevant to hashlib, that would be accelerated by OS
native
Gregory P. Smith added the comment:
New changeset f6b3a07b7df60dc04d0260169ffef6e9796a2124 by ty in branch 'main':
bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866)
https://github.com/python/cpython/commit/f6b3a07b7df60dc04d0260169
Gregory P. Smith added the comment:
New changeset dae09c2b819c2683ad870733451c050b59c3eb93 by Miss Islington (bot)
in branch '3.9':
[3.9] bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866)
(GH-32140) (GH-32156)
https://github.com/python
Change by Gregory P. Smith :
--
resolution: -> fixed
___
Python tracker
<https://bugs.python.org/issue44493>
___
___
Python-bugs-list mailing list
Unsubscrib
Change by Gregory P. Smith :
--
components: +Extension Modules -Library (Lib)
stage: -> needs patch
title: vfork() returns EINVAL if PID=1 -> subprocess fails when used as init,
vfork() returns EINVAL if PID=1
type: -> behavior
___
Pytho
Gregory P. Smith added the comment:
New changeset dc2d8404a3ab6288ce112c71da8c65c34cd3087e by Dave Goncalves in
branch 'main':
bpo-33178: Add BigEndianUnion, LittleEndianUnion classes to ctypes (GH-25480)
https://github.com/python/cpython/commit/dc2d8404a3ab6288ce112c71da8c65
Gregory P. Smith added the comment:
Thanks for the contribution!
(our attention spans can be scattered and random, apologies for the delay)
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracke
Gregory P. Smith added the comment:
yep, branch off of a recent main.
--
___
Python tracker
<https://bugs.python.org/issue44733>
___
___
Python-bugs-list mailin
Change by Gregory P. Smith :
--
keywords: +patch
pull_requests: +30263
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/32186
___
Python tracker
<https://bugs.python.org/issu
Gregory P. Smith added the comment:
Any possibility that you can test the attached PR as pid 1?
--
___
Python tracker
<https://bugs.python.org/issue47
New submission from Gregory P. Smith :
## Background
All `hashlib` computations and `binascii.crc32` and `zlib.crc32` release the
GIL around their computational core. But they use a hard coded length check to
determine when to do so, or always do it.
That already accomplishes the larger
Gregory P. Smith added the comment:
Thanks. I had wondered if this was really a pid=1 restriction or not, but I
could definitely imagine kernel scenarios where vfork is simply forbidden
regardless. There are a variety of policy mechanisms in kernels, mainline Linux
or not, that _could_ do
Gregory P. Smith added the comment:
New changeset 4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835 by Gregory P. Smith in
branch 'main':
bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186)
https://github.com/python/cpython/commit/4a08c4c469d36f99d3a5e0f17ad82a
Change by Gregory P. Smith :
--
assignee: -> gregory.p.smith
resolution: -> fixed
stage: patch review -> commit review
status: open -> closed
___
Python tracker
<https://bugs.python
New submission from Gregory P. Smith :
zlib-ng is an optimized zlib library with better performance on most
architectures (with contributions from the likes of Google, Cloudflare, and
Intel). It is API compatible with zlib. https://github.com/zlib-ng/zlib-ng
I believe the only platform we
New submission from Gregory P. Smith :
zlib v1.2.11 as used in Windows binary releases contains a security issue that,
while fixed in its git repo years ago, never wound up in a release or a CVE
until just now.
Folllow the https://www.openwall.com/lists/oss-security/2022/03/24/1 thread and
Gregory P. Smith added the comment:
A more fundamental issue behind this: unittest's plain text verbose output is
not intended to be parsable and consumable by machines. It is 22 years old and
was intended for use in 80 column wide terminals as a progress report while
watching it r
Gregory P. Smith added the comment:
per
https://mail.python.org/archives/list/python-...@python.org/message/GFOMU7LP63JUVFMWNJNZJLUMZDRPTUYJ/
lets roll some of those PRs back and reshape where we move these APIs as
despite the PEP they were not as private as other privates.
(deferred
Gregory P. Smith added the comment:
If we can conditionally test new things based on C++XX version, accumulating
modern issue regression tests seems useful. Otherwise 11 at minimum.
As for why some things trigger this and others don't, my wild _guess_ would be
whether the statements a
Change by Gregory P. Smith :
--
nosy: +gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue47185>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Gregory P. Smith :
--
nosy: +gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue47164>
___
___
Python-bugs-list mailing list
Unsubscribe:
Gregory P. Smith added the comment:
The irony... Documenting the caveat at least seems useful. Your workaround
sounds reasonable.
I don't love the idea of implementing our own mask blocked/unblocked state
check, though it probably wouldn't be very complicated. Might be interesting
Change by Gregory P. Smith :
--
nosy: +gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue47153>
___
___
Python-bugs-list mailing list
Unsubscribe:
Gregory P. Smith added the comment:
New changeset 75280944e5ca957eec7f814b9d0608fc84fc5811 by Hugo van Kemenade in
branch 'main':
bpo-46607: Add DeprecationWarning for LegacyInterpolation, deprecated in docs
since 3.2 (GH-30927)
https://github.com/python/cpyt
Gregory P. Smith added the comment:
The "trick" wouldn't be too useful though as this API can't block and the
signal flag needs to be processed on the main thread. So I guess documentation
it is.
The way I think of this is that the signal.pthread_sigmask API is pretty lo
Gregory P. Smith added the comment:
this mostly requires plumbing to accept an int as the cwd and plumb that
through to the between fork and exec code to call `fchdir(cwd_fd)` on the `int`
instead of chdir(cwd) on the `char*`.
the Modules/_posixsubprocess.c internals are a bit of a mess
Gregory P. Smith added the comment:
LOL... we didn't anticipate this happening when adding the commonly found typo
detection feature.
--
assignee: docs@python -> gregory.p.smith
nosy: +gregory.p.smith
___
Python tracker
<https://bugs
Change by Gregory P. Smith :
--
keywords: +patch
pull_requests: +30401
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/32348
___
Python tracker
<https://bugs.python.org/issu
Gregory P. Smith added the comment:
New changeset ac1fb07b6ecb6b93446484f52894914e5199de63 by Gregory P. Smith in
branch 'main':
bpo-47235: Note where a typo is intentional in code. (GH-32348)
https://github.com/python/cpython/commit/ac1fb07b6ecb6b93446484f5289491
Change by Gregory P. Smith :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: -Python 3.8
___
Python tracker
<https://bugs.python.or
Gregory P. Smith added the comment:
New changeset 612e422c6ea9df60d3382ed1491d8254c283e93f by Jeremy Kloth in
branch 'main':
bpo-46576: Speed up test_peg_generator by using a static library for shared
sources (GH-32338)
https://github.com/python/cpyt
Gregory P. Smith added the comment:
Can you provide a reproducable way to demonstrate evidence of a problem in
CPython's use of the Linux libc vfork() implementation? A test case that
causes a CPython parent or child process on Linux when built with HAVE_VFORK
failing to function pro
Gregory P. Smith added the comment:
Immediate action item: Add a way for people to disable vfork at runtime by
setting a flag in the subprocess module, just in case.
This can be backported to 3.10 - It'd provide an escape hatch for anyone
without a need to rebuild Python to disable u
Change by Gregory P. Smith :
--
keywords: +3.10regression
___
Python tracker
<https://bugs.python.org/issue47245>
___
___
Python-bugs-list mailing list
Unsub
Gregory P. Smith added the comment:
Our current assumptions around the use of vfork() are very much glibc specific.
Another useful reference for reasoning, comments, and history is
https://github.com/golang/go/blob/master/src/syscall/exec_linux.go#L146
`forkAndExecInChild1
Gregory P. Smith added the comment:
Thanks! I agree with you that this is probably not an actual problem on Linux.
_I did look at the various glibc architecture vfork.s implementations: Cute
tricks used on some where they need to avoid a stack modifying traditional
return from vfork
Gregory P. Smith added the comment:
https://discuss.python.org/t/need-reconsideration-of-bpo-34975-add-start-tls-method-to-streams-api/14720
would like to see this reconsidered. reopening.
--
nosy: +gregory.p.smith
resolution: wont fix ->
stage: resolved ->
status: closed -
Gregory P. Smith added the comment:
New changeset 1c8b3b5d66a629258f1db16939b996264a8b9c37 by Alexey Izbyshev in
branch 'main':
bpo-47260: Fix os.closerange() potentially being a no-op in a seccomp sandbox
(GH-32418)
https://github.com/python/cpyt
Gregory P. Smith added the comment:
Good catch.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Gregory P. Smith added the comment:
Nope, work remains to be done. I've got an msan buildbot system waiting but
haven't had time to follow up on figuring out what remains in a while.
(getting a functioning memory sanitizer build is... finnicky to say
Gregory P. Smith added the comment:
I'm not a fan of CVE numbers in general, people have been creating too many of
those. But that also means I just don't care if someone does. Having a CVE
entry is not a way to claim something is important.
This issue is still open and can
New submission from Gregory P. Smith :
test_logging hangs when run on an IPv6-only host. (127.0.0.1 isn't even
available)
test_listen_config_10_ok (test.test_logging.ConfigDictTest) ... Exception in
thread Thread-3:
Traceback (most recent call last):
File "/home/greg/oss/c
New submission from Gregory P. Smith :
It winds up stuck on a leftover process:
test_import (test.test_multiprocessing_fork._TestImportStar) ... ok
Warning -- Dangling processes: {}
Which is likely related to one of the other numerous failure ERRORs further up
in the log.
I'm prepari
New submission from Gregory P. Smith :
test_drain_raises (test.test_asyncio.test_streams.StreamTests) ... Exception in
thread Thread-20:
Traceback (most recent call last):
File "/home/greg/oss/cpython/Lib/threading.py", line 938, in _bootstrap_inner
self.run()
File "
New submission from Gregory P. Smith :
test_err (test.test_httpservers.RequestHandlerLoggingTestCase) ... Exception in
thread Thread-1:
Traceback (most recent call last):
File "/home/greg/oss/cpython/Lib/threading.py", line 938, in _bootstrap_inner
self.run()
File "
New submission from Gregory P. Smith :
test_404 (test.test_xmlrpc.SimpleServerTestCase) ... Exception in thread
Thread-1:
Traceback (most recent call last):
File "/home/greg/oss/cpython/Lib/threading.py", line 938, in _bootstrap_inner
self.run()
File "/home/greg/
New submission from Gregory P. Smith :
21 tests failed:
test_asynchat test_asyncore test_docxmlrpc test_eintr test_epoll
test_ftplib test_httplib test_imaplib
test_multiprocessing_forkserver test_multiprocessing_spawn
test_nntplib test_os test_poplib test_robotparser
Change by Gregory P. Smith :
--
versions: +Python 3.9 -Python 3.8
___
Python tracker
<https://bugs.python.org/issue32949>
___
___
Python-bugs-list mailin
Change by Gregory P. Smith :
--
versions: +Python 3.9 -Python 3.7
___
Python tracker
<https://bugs.python.org/issue29988>
___
___
Python-bugs-list mailin
Gregory P. Smith added the comment:
The signal handler in this case is CPython's internal signal handling system
thus any such onus falls on us...
The great signal handling hack of "set a flag that the interpreter loop checks
on occasion" trick lasted a long time, but o
Gregory P. Smith added the comment:
FWIW, _posixsubprocess.fork_exec() should be safe to allow.
The only thing within it to disallow, if you're going to bother to check this
at all, is any use of the legacy preexec_fn support.
--
nosy: +gregory.p.
Gregory P. Smith added the comment:
META: Do we still have platforms we support that we do not also require a
modern enough OpenSSL version in order for CPython to build that'd provide
blake2 for us?
I'd love to ditch maintenance of our own copies of the various secure hash
Gregory P. Smith added the comment:
oh well, no pony for us this decade. :)
--
___
Python tracker
<https://bugs.python.org/issue37055>
___
___
Python-bugs-list m
Change by Gregory P. Smith :
--
versions: +Python 3.8 -Python 3.4
___
Python tracker
<https://bugs.python.org/issue13788>
___
___
Python-bugs-list mailin
Gregory P. Smith added the comment:
I believe this has been addressed.
--
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/i
Change by Gregory P. Smith :
--
nosy: +benjamin.peterson
___
Python tracker
<https://bugs.python.org/issue1652>
___
___
Python-bugs-list mailing list
Unsub
Change by Gregory P. Smith :
--
pull_requests: +15502
pull_request: https://github.com/python/cpython/pull/15853
___
Python tracker
<https://bugs.python.org/issue28
Change by Gregory P. Smith :
--
versions: +Python 3.9
___
Python tracker
<https://bugs.python.org/issue38047>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Gregory P. Smith :
https://github.com/python/cpython/pull/15274
--
assignee: gregory.p.smith
messages: 351700
nosy: gregory.p.smith
priority: normal
severity: normal
status: open
title: unneeded assignment to wb.len in PyBytes_Concat using buffer protocol
versions
Gregory P. Smith added the comment:
Shai, please open the 'Your Details' link in the bugs.python.org sidebar and
make sure you have your github username filled in. it needs to say ShaiAvr for
our CLA automation to understand.
Avoiding calling list() on the output of scandir()
Gregory P. Smith added the comment:
The new ziptestdata/ subdir appears to not be part of the install that make
install does. :/
--
___
Python tracker
<https://bugs.python.org/issue28
Gregory P. Smith added the comment:
For some context here, the pathlib scandir code was written by Serhiy in
https://bugs.python.org/issue26032 and related commit
https://github.com/python/cpython/commit/680cb152c5d220a74321fa905d4fc91bdec40fbb.
> Any optimization can be accepted only w
Change by Gregory P. Smith :
--
pull_requests: +15543
pull_request: https://github.com/python/cpython/pull/15902
___
Python tracker
<https://bugs.python.org/issue28
Gregory P. Smith added the comment:
New changeset 7a0023e8d17566eb32c836b65c33663303a2224f by Gregory P. Smith
(smokephil) in branch 'master':
bpo-25810: Clarify eval() docs, it does not keywords (GH-15173)
https://github.com/python/cpython/commit/7a0023e8d17566eb32c836b65c3366
Gregory P. Smith added the comment:
New changeset c37447481ec8f6d0e49d0587ec0de3f9e7d56b28 by Gregory P. Smith in
branch 'master':
bpo-28494: install ziptestdata to fix install bot (GH-15902)
https://github.com/python/cpython/commit/c37447481ec8f6d0e49d0587ec0de3
New submission from Gregory P. Smith :
On POSIX platforms that support the fdwalk() API, use it within the
os.closerange() implementation.
--
components: Demos and Tools
messages: 351875
nosy: gregory.p.smith
priority: normal
severity: normal
stage: patch review
status: open
title
Gregory P. Smith added the comment:
contribution from Jakub Kulík (see the PR)
--
assignee: -> gregory.p.smith
___
Python tracker
<https://bugs.python.org/issu
Gregory P. Smith added the comment:
Thanks. I believe this issue is fixed but you've identified follow-on issues.
lets follow up on those in their own bugs.
--
resolution: -> fixed
stage: patch review -> commit review
status: ope
Change by Gregory P. Smith :
--
priority: normal -> low
___
Python tracker
<https://bugs.python.org/issue37091>
___
___
Python-bugs-list mailing list
Un
Gregory P. Smith added the comment:
"""On Solaris, we are patching (for many years now) `posix_closerange` function
to use `fdwalk` to close file descriptors instead of a for loop. While for a
long time only Solaris had `fdwalk`, but if I am not mistaken, some other OSes
imp
Gregory P. Smith added the comment:
New changeset e20134f889a0cfcc37a46979f31a1c98b800de07 by Gregory P. Smith
(Jakub Kulík) in branch 'master':
bpo-38110: Use fdwalk for os.closerange() when available. (GH-15224)
https://github.com/python/cpyt
Change by Gregory P. Smith :
--
assignee: -> michael.foord
___
Python tracker
<https://bugs.python.org/issue36871>
___
___
Python-bugs-list mailing list
Un
Change by Gregory P. Smith :
--
resolution: fixed ->
stage: resolved -> needs patch
status: closed -> open
___
Python tracker
<https://bugs.python.or
Gregory P. Smith added the comment:
i reopened this without diving into the code to better understand based on
Samuels comment.
We could really do with a testcase that demonstrates the misleading error
message problem for some test driven development here.
--
keywords: -patch
Gregory P. Smith added the comment:
New changeset 2c910c1e732c9a3ec4c67a7c43d789d6c729304a by Gregory P. Smith
(Julien Palard) in branch 'master':
bpo-36675: Remove obsolete code. (GH-16024)
https://github.com/python/cpython/commit/2c910c1e732c9a3ec4c67a7c43d789d6c729304a
-
Gregory P. Smith added the comment:
New changeset 64117e059b79236c7345bc9afc1cc707162411de by Gregory P. Smith
(Christian Heimes) in branch 'master':
bpo-38134: Remove PKBDF2_HMAC_fast from _hashopenssl (GH-16028)
https://github.com/python/cpyt
Gregory P. Smith added the comment:
New changeset f9dc2ad89032201427ed5f08061c703794627ad9 by Gregory P. Smith
(Serhiy Storchaka) in branch 'master':
bpo-37935: Added tests for os.walk(), glob.iglob() and Path.glob() (GH-15956)
https://github.com/python/cpyt
801 - 900 of 3274 matches
Mail list logo
