Re: [Python-Dev] USE_STACKCHECK and running out of stack

2018-07-30 Thread Gregory P. Smith
On Sat, Jul 28, 2018 at 7:51 AM Ronald Oussoren via Python-Dev < python-dev@python.org> wrote: > Hi, > > I’m looking at PyOS_CheckStack because this feature might be useful on > macOS (and when I created bpo-33955 for this someone ran with it and > created a patch). > > Does anyone remember why th

Re: [Python-Dev] AES decryption/encryption support for zipfile

2018-09-10 Thread Gregory P. Smith
On Wed, Sep 5, 2018 at 8:24 AM 大野隆弘 wrote: > Sorry, allow me to ask one more thing. > If I want to use AES in zipfile module, what the good way to implement? > If anyone wants to add support for additional zipfile encryption/decryption methods, there are a few options: (a) Fork the stdlib zipfi

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

2018-09-19 Thread Gregory P. Smith
On Sat, Sep 15, 2018 at 2:53 AM Paul Moore wrote: > On Fri, 14 Sep 2018 at 23:28, Neil Schemenauer > wrote: > > > > On 2018-09-14, Larry Hastings wrote: > > > [..] adding the stat calls back in costs you half the startup. So > > > any mechanism where we're talking to the disk _at all_ simply >

Re: [Python-Dev] Questions about signal handling.

2018-09-25 Thread Gregory P. Smith
On Mon, Sep 24, 2018 at 1:20 PM Eric Snow wrote: > On Mon, Sep 24, 2018 at 11:14 AM Yury Selivanov > wrote: > > On Fri, Sep 21, 2018 at 7:04 PM Eric Snow > wrote: > > > 1. Why do we restrict calls to signal.signal() to the main thread? > > > 2. Why must signal handlers run in the main thread? >

Re: [Python-Dev] Use of objdump within ctypes _get_soname()

2018-10-09 Thread Gregory P. Smith
On Mon, Oct 8, 2018 at 11:59 PM Ray Donnelly wrote: > Hi, > > We ran into an issue on the Anaconda Distribution recently where we > added libarchive-c to conda-build (so we can un-compress more source > archive formats than tarfile supports) and everything was good a few > hours, until it hit var

Re: [Python-Dev] "Deprecation" of os.system in favor of subprocess?

2018-10-24 Thread Gregory P. Smith
The os module is by definition special. It exposes libc and platform APIs. That there are Python modules that provide similar functionality, often surpassing it and sometimes being built on top of it, is intentional. Random quotes from the Zen don't win arguments. Although practicality beats pu

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-12 Thread Gregory P. Smith
On Fri, Nov 9, 2018 at 5:50 PM Nathaniel Smith wrote: > On Fri, Nov 9, 2018 at 4:30 PM, Victor Stinner > wrote: > > Ah, important points. I don't want to touch the current C API nor make > > it less efficient. And compatibility in both directions (current C API > > <=> new C API) is very importa

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-12 Thread Gregory P. Smith
On Sun, Nov 11, 2018 at 3:19 PM Victor Stinner wrote: > Le sam. 10 nov. 2018 à 04:02, Nathaniel Smith a écrit : > > So is it fair to say that your plan is that CPython will always use > > the current ("old") API internally, and the "new" API will be > > essentially an abstraction layer, that's d

Re: [Python-Dev] General concerns about C API changes

2018-11-14 Thread Gregory P. Smith
On Tue, Nov 13, 2018 at 7:06 PM Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > Overall, I support the efforts to improve the C API, but over the last few > weeks have become worried. I don't want to hold up progress with fear, > uncertainty, and doubt. Yet, I would like to be more com

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

2018-11-14 Thread Gregory P. Smith
> > It seems like the discussion so far is: > > Victor: "I know people when people hear 'new API' they get scared and > think we're going to do a Python-3-like breaking transition, but don't > worry, we're never going to do that." > Nathaniel: "But then what does the new API add?" > Greg: "It lets

Re: [Python-Dev] C API changes

2018-11-27 Thread Gregory P. Smith
On Mon, Nov 26, 2018 at 4:10 PM Larry Hastings wrote: > On 11/23/18 5:15 AM, Armin Rigo wrote: > > Also FWIW, my own 2 cents on the topic of changing the C API: let's > entirely drop ``PyObject *`` and instead use more opaque > handles---like a ``PyHandle`` that is defined as a pointer-sized C >

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-11-28 Thread Gregory P. Smith
On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon wrote: > Are we getting to the point that we want a compresslib like hashlib if we > are going to be adding more compression algorithms? > Lets avoid the lib suffix when unnecessary. I used the name hashlib because the name hash was already taken by

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-11-29 Thread Gregory P. Smith
On Wed, Nov 28, 2018 at 10:43 AM Gregory P. Smith wrote: > > On Wed, Nov 28, 2018 at 9:52 AM Brett Cannon wrote: > >> Are we getting to the point that we want a compresslib like hashlib if we >> are going to be adding more compression algorithms? >> > &g

Re: [Python-Dev] Standard library vs Standard distribution?

2018-11-29 Thread Gregory P. Smith
meta: I'm not participating in this sub-thread. Just changing the subject line. On Thu, Nov 29, 2018 at 9:17 AM Christian Heimes wrote: > On 29/11/2018 17.32, Antoine Pitrou wrote: > > We may ask ourselves if there is really a large difference between a > > "standard distribution" and a "standar

Re: [Python-Dev] Inclusion of lz4 bindings in stdlib?

2018-11-29 Thread Gregory P. Smith
2018 at 12:04 PM INADA Naoki > wrote: > >> On Thu, Nov 29, 2018 at 6:27 AM Steven D'Aprano >> wrote: >> > >> > On Wed, Nov 28, 2018 at 10:43:04AM -0800, Gregory P. Smith wrote: >> > >> > > PyPI makes getting more algorithms easy. &

Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

2019-01-17 Thread Gregory P. Smith
I've heard that libraries using ctypes, cffi, or cython code of various sorts in the real world wild today does abuse the unfortunate side effect of CPython's implementation of id(). I don't have specific instances of this in mind but trust what I've heard: that it is happening. id() should never

Re: [Python-Dev] Sub-interpreters: importing numpy causes hang

2019-01-27 Thread Gregory P. Smith
On Thu, Jan 24, 2019 at 1:25 PM Chris Barker - NOAA Federal via Python-Dev < python-dev@python.org> wrote: > If your primary concern is module clashes between plugins, maybe you > can hack around that: > > 1) if the plugins are providing copies of any other modules, then you > can simply require t

Re: [Python-Dev] How to update namedtuple asdict() to use dict instead of OrderedDict

2019-01-30 Thread Gregory P. Smith
On Wed, Jan 30, 2019 at 2:32 PM Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > Now that regular dicts are ordered and compact, it makes more sense for > the _asdict() method to create a regular dict (as it did in its early days) > rather than an OrderedDict. The regular dict is much sm

Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform

2019-02-14 Thread Gregory P. Smith
On Thu, Feb 14, 2019 at 11:38 AM Steve Dower wrote: > As part of adding ARM32 support for Windows, we need to enable > cross-compilation in distutils. This is easy enough, though it requires > somehow getting the target platform as well as the current platform. > > Right now, the change at https:

Re: [Python-Dev] Is distutils.util.get_platform() the "current" or the "target" platform

2019-02-15 Thread Gregory P. Smith
On Fri, Feb 15, 2019 at 2:02 PM Steve Dower wrote: > On 14Feb2019 1147, Gregory P. Smith wrote: > > To alleviate confusion long term I'd love it if we could deprecate the > > unqualified get_platform() API and point people towards always being > > explicit abou

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-15 Thread Gregory P. Smith
On Thu, Feb 14, 2019 at 9:29 AM Barry Warsaw wrote: > On Feb 13, 2019, at 23:08, Matěj Cepl wrote: > > > Is this relevant to the discussion at hand? We are talking about > > the binary /usr/bin/python3 which will be surely be provided > > even by Python 4, won't it? > > Why would it be? Since t

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-18 Thread Gregory P. Smith
On Mon, Feb 18, 2019, 7:34 AM Matthias Klose On 16.02.19 00:15, Gregory P. Smith wrote: > > On Thu, Feb 14, 2019 at 9:29 AM Barry Warsaw wrote: > > > >> On Feb 13, 2019, at 23:08, Matěj Cepl wrote: > >> > >>> Is this relevant to the discussion at hand?

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-26 Thread Gregory P. Smith
On Tue, Feb 26, 2019 at 8:59 AM INADA Naoki wrote: > > > > With *Homebrew*, `python` points to Homebrew’s Python 2.7.x (if > > installed) otherwise the macOS system Python. That's exactly according > > to the PEP. They tried to switch python to python3 before, and got > > rather nasty backlash ci

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-26 Thread Gregory P. Smith
On Tue, Feb 26, 2019 at 9:55 AM Barry Warsaw wrote: > > There haven't been many new ideas since this summary – mostly it was > explaining and re-hashing what's been mentioned before. > > Thanks for the summary Petr. > > Here’s another way to think about the problem. I know Nick and I have > talk

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-27 Thread Gregory P. Smith
On Tue, Feb 26, 2019 at 2:28 PM Victor Stinner wrote: > Le mar. 26 févr. 2019 à 22:24, Gregory P. Smith a écrit > : > > A feature that I find missing from posix-y OSes that support #! lines is > an ability to restrict what can use a given interpreter. > > Fedora runs sys

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

2019-02-27 Thread Gregory P. Smith
On Tue, Feb 26, 2019 at 2:31 PM Steve Dower wrote: > On 2/26/2019 1:20 PM, Gregory P. Smith wrote: > > For an OS distro provided interpreter, being able to restrict its use to > > only OS distro provided software would be ideal (so ideal that people > > who haven'

Re: [Python-Dev] Compile-time resolution of packages [Was: Another update for PEP 394...]

2019-02-28 Thread Gregory P. Smith
On Wed, Feb 27, 2019 at 5:12 PM Toshio Kuratomi wrote: > > On Tue, Feb 26, 2019 at 2:07 PM Neil Schemenauer > wrote: > >> On 2019-02-26, Gregory P. Smith wrote: >> > On Tue, Feb 26, 2019 at 9:55 AM Barry Warsaw wrote: >> > For an OS distro provided interpr

Re: [Python-Dev] PEP 581: Using GitHub Issues for CPython

2019-03-07 Thread Gregory P. Smith
On Thu, Mar 7, 2019 at 2:12 PM Mariatta wrote: > > On Thu, Mar 7, 2019 at 12:35 PM Matthew Woodcraft > wrote: > >> >> One part of this PEP stands out to me: >> >> | We should not be moving all open issues to GitHub. Issues with little >> | or no activity should just be closed. Issues with no dec

Re: [Python-Dev] (Licensing question) backport of shutil.copyfile() functionality

2019-03-11 Thread Gregory P. Smith
Things in the standard library are already covered by the PSF license so that is what should be kept on backports from the stdlib to earlier versions. I do recommend keeping your backported stuff and new functionality separate (separate packages ideally, but that'll depend on how intertwined thing

Re: [Python-Dev] (Licensing question) backport of shutil.copyfile() functionality

2019-03-12 Thread Gregory P. Smith
On Tue, Mar 12, 2019 at 2:55 PM Giampaolo Rodola' wrote: > > > On Tue, Mar 12, 2019 at 3:01 AM Glenn Linderman > wrote: > >> On 3/11/2019 4:35 PM, Giampaolo Rodola' wrote: >> >> Hello, >> some time ago I contributed a couple of patches to speedup shutil.copy*() >> functions: >> https://bugs.pyth

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-18 Thread Gregory P. Smith
On Mon, Mar 18, 2019 at 9:44 PM Terry Reedy wrote: > On 3/18/2019 6:41 PM, Raymond Hettinger wrote: > > We're having a super interesting discussion on > https://bugs.python.org/issue34160 . It is now marked as a release > blocker and warrants a broader discussion. > > > > Our problem is that at

Re: [Python-Dev] Is XML serialization output guaranteed to be bytewise identical forever?

2019-03-19 Thread Gregory P. Smith
On Tue, Mar 19, 2019, 4:53 AM Ned Batchelder wrote: > On 3/19/19 4:13 AM, Serhiy Storchaka wrote: > > 19.03.19 00:41, Raymond Hettinger пише: > >> 4) Fix the tests in the third-party modules to be more focused on > >> their actual test objectives, the semantics of the generated XML > >> rather th

Re: [Python-Dev] Best way to specify docstrings for member objects

2019-03-20 Thread Gregory P. Smith
(answers above and below the quoting) I like the idea of documenting attributes, but we shouldn't force the user to use __slots__ as that has significant side effects and is rarely something people should bother to use. There are multiple types of attributes. class and instance. but regardless

[Python-Dev] PEP 556 threaded garbage collection & linear recursion in gc

2019-03-27 Thread Gregory P. Smith
https://www.python.org/dev/peps/pep-0556/ This PEP is currently Deferred as nobody is actively working on a test implementation. A situation came up the other day where I *believe* this could've helped. Scenario (admittedly not one most environments run into): A Python process with a C++ extensi

Re: [Python-Dev] PEP 556 threaded garbage collection & linear recursion in gc

2019-03-27 Thread Gregory P. Smith
ual next time I hear of such a report. -G On Wed, Mar 27, 2019, 5:38 PM Tim Peters wrote: > [Gregory P. Smith ] > > ... > > A situation came up the other day where I believe this could've helped. > > > > Scenario (admittedly not one most environments run into)

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-03-30 Thread Gregory P. Smith
I wouldn't expect it to be the case in a CI environment but I believe a umask can be overridden if the filesystem is mounted and configured with acls set? (oh, hah, Ivan just said the same thing) -gps On Sat, Mar 30, 2019 at 9:05 AM Steve Dower wrote: > On 29Mar.2019 1944, Steve Dower wrote: >

Re: [Python-Dev] Strange umask(?)/st_mode issue

2019-04-01 Thread Gregory P. Smith
On Mon, Apr 1, 2019 at 4:49 PM Cameron Simpson wrote: > On 01Apr2019 15:44, Steve Dower wrote: > >On 01Apr2019 1535, Cameron Simpson wrote: > >>On 01Apr2019 09:12, Steve Dower wrote: > >>>On 30Mar2019 1130, Gregory P. Smith wrote: > >>>>I wouldn&#

Re: [Python-Dev] Need help to fix HTTP Header Injection vulnerability

2019-04-09 Thread Gregory P. Smith
On Tue, Apr 9, 2019 at 4:45 PM Karthikeyan wrote: > I would recommend fixing it since it's potentially remote code execution > on systems like Redis (latest versions of Redis have this mitigated) though > I must admit I don't fully understand the complexity since there are > multiple issues linke

Re: [Python-Dev] Need help to fix HTTP Header Injection vulnerability

2019-04-10 Thread Gregory P. Smith
On Wed, Apr 10, 2019 at 11:00 AM Ivan Pozdeev via Python-Dev < python-dev@python.org> wrote: > > On 10.04.2019 7:30, Karthikeyan wrote: > > Thanks Gregory. I think it's a good tradeoff to ensure this validation > only for URLs of http scheme. > > I also agree handling newline is little problematic

Re: [Python-Dev] Unicode identifiers in test files?

2019-05-03 Thread Gregory P. Smith
On Fri, May 3, 2019 at 10:46 PM Eric V. Smith wrote: > Is there a policy against using Unicode identifiers in test files? > > As part of adding !d to f-strings, there's a code path that's only > executed if the text of the expression is non-ascii. The easiest way to > exercise it, and the way I f

Re: [Python-Dev] bpo-36829: Add sys.unraisablehook()

2019-05-16 Thread Gregory P. Smith
I like the feature, we should have it. It'll be useful for debugging and probably more. Which brings me to the annoying paint color question: These exceptions were most definitely raised. Thus the term "unraisable" is wrong. I believe you really mean "uncatchable". -gps On Thu, May 16, 2019 at

Re: [Python-Dev] Have a big machine and spare time? Here's a possible Python bug.

2019-05-23 Thread Gregory P. Smith
I suggest filing a bug to track this... On Thu, May 23, 2019 at 10:13 AM Tim Peters wrote: > [Inada Naoki ] > > ... > > 2. This loop is cleary hot: > > > https://github.com/python/cpython/blob/51aa35e9e17eef60d04add9619fe2a7eb938358c/Objects/obmalloc.c#L1816-L1819 > > Which is 3 lines of code pl

Re: [Python-Dev] [python-committers] PEP 595: Improving bugs.python.org

2019-05-24 Thread Gregory P. Smith
-cc: committers to avoid crossposting. I have feedback for roundup as experienced on BPO that should be represented within PEP 595 if we are going to have a summary of "improving roundup for BPO" captured in a PEP (presumably already rejected given 581? But good to have documented regardless so _t

Re: [Python-Dev] [python-committers] PEP 595: Improving bugs.python.org

2019-05-24 Thread Gregory P. Smith
On Fri, May 24, 2019 at 1:48 PM Ezio Melotti wrote: > > On Fri, May 24, 2019, 20:23 Gregory P. Smith wrote: > >> -cc: committers to avoid crossposting. >> > > +1 (I wanted to include committers, since the announcement about PEP 581 > was posted there too, but it&#

[Python-Dev] Re: python-ideas and python-dev migrated to Mailman 3/HyperKitty

2019-06-07 Thread Gregory P. Smith
On Thu, Jun 6, 2019 at 6:19 AM Victor Stinner wrote: > Le jeu. 6 juin 2019 à 14:18, Steven D'Aprano a > écrit : > > i.e. 25-40% longer. Is there a shorter permalink form available, like > > goo.gl, bitly, youtu.be etc use? That would be awesome if we could use > > them instead. > > I really disl

[Python-Dev] Re: PEP 581 has been updated with "Downsides of GitHub" section

2019-07-02 Thread Gregory P. Smith
On Mon, Jul 1, 2019 at 9:01 AM Steven D'Aprano wrote: > On Sat, Jun 29, 2019 at 10:26:04AM -0500, Skip Montanaro wrote: > > > You have missed at least one: the minimum technology requirement for > > > using Github is a lot more stringent than for Roundup. Github's minimum > > > system requirement

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-06 Thread Gregory P. Smith
On Tue, Aug 6, 2019 at 10:06 AM Neil Schemenauer wrote: > > Making it an error so soon would be mistake, IMHO. That will break > currently working code for small benefit. When Python was a young > language with a few thousand users, it was easier to make these > kinds of changes. Now, we shoul

[Python-Dev] Re: What to do about invalid escape sequences

2019-08-06 Thread Gregory P. Smith
People distribute code via pypi. if we reject uploads of packages with these problems and link to fixers (modernize can be taught what to do), we prevent them from spreading further. A few years after doing that, we can revisit how much pain and for whom making this a SyntaxWarning or even Syntax

[Python-Dev] Re: os.scandir bug in Windows?

2020-10-17 Thread Gregory P. Smith
Could you please file this as an issue on bugs.python.org? Thanks! -Greg On Sat, Oct 17, 2020 at 7:25 PM Rob Cliffe via Python-Dev < python-dev@python.org> wrote: > > TLDR: In os.scandir directory entries, atime is always a copy of mtime > rather than the actual access time. > > Demo program: W

[Python-Dev] Re: os.scandir bug in Windows?

2020-10-19 Thread Gregory P. Smith
On Mon, Oct 19, 2020 at 6:28 AM Ivan Pozdeev via Python-Dev < python-dev@python.org> wrote: > > On 19.10.2020 14:47, Steve Dower wrote: > > On 19Oct2020 1242, Steve Dower wrote: > >> On 15Oct2020 2239, Rob Cliffe via Python-Dev wrote: > >>> TLDR: In os.scandir directory entries, atime is always a

[Python-Dev] Re: Speeding up CPython

2020-10-20 Thread Gregory P. Smith
On Tue, Oct 20, 2020 at 5:59 AM Mark Shannon wrote: > Hi everyone, > > CPython is slow. We all know that, yet little is done to fix it. > > I'd like to change that. > I have a plan to speed up CPython by a factor of five over the next few > years. But it needs funding. > > I am aware that there h

[Python-Dev] Re: Speeding up CPython

2020-10-21 Thread Gregory P. Smith
On Wed, Oct 21, 2020 at 4:04 PM Greg Ewing wrote: > A concern I have about this is what effect it will have on the > complexity of CPython's implementation. > > CPython is currently very simple and straightforward. Some parts > are not quite as simple as they used to be, but on the whole it's > f

[Python-Dev] Re: os.scandir bug in Windows?

2020-10-26 Thread Gregory P. Smith
On Mon, Oct 26, 2020, 4:06 PM Chris Angelico wrote: > On Tue, Oct 27, 2020 at 10:00 AM Greg Ewing > wrote: > > > > On 27/10/20 8:24 am, Victor Stinner wrote: > > > I would > > > rather want to kill the whole concept of "access" time in operating > > > systems (or just configure the OS to not upd

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-29 Thread Gregory P. Smith
I agree, remove Solaris support. Nobody willing to contribute seems interested. -gps -- blame half the typos on my phone. On Thu, Oct 29, 2020, 2:50 PM Victor Stinner wrote: > Hi, > > I propose to drop the Solaris support in Python to reduce the Python > maintenance burden: > >https://bugs

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-30 Thread Gregory P. Smith
On Fri, Oct 30, 2020 at 8:26 AM Sebastian Wiedenroth wrote: > Hi, > > I've already commented on the issue, but want to make a few more points > here as well. > > Removing Solaris support would not only impact Oracle Solaris but the open > source illumos community as well. > Both systems share the

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-30 Thread Gregory P. Smith
On Fri, Oct 30, 2020 at 11:03 AM Brett Cannon wrote: > > On Fri, Oct 30, 2020 at 6:37 AM Pablo Galindo Salgado > wrote: > >> >Two volunteer core developers and at least one buildbot would help a >> > lot to ensure that Python is working on Solaris for real, and reduce >> > the number of open Sol

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-30 Thread Gregory P. Smith
On Fri, Oct 30, 2020 at 1:14 PM Raymond Hettinger < raymond.hettin...@gmail.com> wrote: > FWIW, when the tracker issue landed with a PR, I became concerned that it > would be applied without further discussion and without consulting users. An issue and a PR doesn't simply mean "it is happening".

[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-30 Thread Gregory P. Smith
On Fri, Oct 30, 2020 at 2:30 PM Garrett D'Amore via Python-Dev < python-dev@python.org> wrote: > I’m not on this list. But I have offered to help - if there are tasks > that need to be done to help this I can help put the weight of a commercial > entity behind it whether that involves assigning o

[Python-Dev] Re: nanosecond stat fields, but not os.path methods ?

2020-12-07 Thread Gregory P. Smith
If the precision is available via OS APIs, this is mostly an issue+PR away from being implemented by someone who cares. FAT32 has a two billion nanosecond resolution IIRC. :P -gps On Mon, Dec 7, 2020 at 8:22 AM David Mertz wrote: > Are there any filesystems that can actually record a meaningfu

[Python-Dev] Re: __future__ annotations loses closure scope

2020-12-08 Thread Gregory P. Smith
What is the utility of a type annotation when the thing it refers to cannot exist? Deferred annotation lookups are intended to be something that analysis time can make sense of but can always have no useful meaning at runtime. No nesting required: ``` from __future__ import annotations Class X:

[Python-Dev] Re: macOS issues with 3.8.7rc1

2020-12-09 Thread Gregory P. Smith
As a meta question: Is there a good reason to support binaries running on macOS earlier than ~ $latest_version-1? Aren't systems running those old releases rather than upgrading unsupported by Apple, never to be patched, and thus not wise to even have on a network? Yes, that means some very old h

[Python-Dev] Re: unittest of sequence equality

2020-12-22 Thread Gregory P. Smith
Numpy chose to violate the principal of equality by having __eq__ not return a bool. So a numpy type can't be used reliably outside of the numpy DSL. -gps On Tue, Dec 22, 2020, 11:51 AM Alan G. Isaac wrote: > Here, `seq1 == seq2` produces a boolean array (i.e., an array of boolean > values). >

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-13 Thread Gregory P. Smith
My take on this is to keep it simple for CPython: Require the newer version of Sphinx on all branches we backport docs changes to. We, the CPython project, determine the exact version(s) of Sphinx a documentation build for a given version of CPython requires. If Sphinx has unfortunately made a br

[Python-Dev] Re: Why aren't we allowing the use of C11?

2021-01-28 Thread Gregory P. Smith
On Thu, Jan 28, 2021 at 10:52 AM Charalampos Stratakis wrote: > > > - Original Message - > > From: "Mark Shannon" > > To: "Python Dev" > > Sent: Thursday, January 28, 2021 5:26:37 PM > > Subject: [Python-Dev] Why aren't we allowing the use of C11? > > > > Hi everyone, > > > > PEP 7 says

[Python-Dev] Re: Security releases of CPython

2021-02-12 Thread Gregory P. Smith
On Thu, Feb 11, 2021 at 8:29 PM Terry Reedy wrote: > On 2/11/2021 3:23 PM, Michał Górny wrote: > > Hello, > > > > I'm the primary maintainer of CPython packages in Gentoo. I would like > > to discuss possible improvement to the release process in order to > > accelerate releasing security fixes t

[Python-Dev] Re: PEP 647 (type guards) -- final call for comments

2021-02-12 Thread Gregory P. Smith
My primary reaction seems similar to Mark Shannon's. When I see this code: def is_str_list(val: List[object]) -> TypeGuard[List[str]]: ... I cannot tell what it returns. There is no readable indication in that this returns a boolean so the reader cannot immediately see how to use the functi

[Python-Dev] Re: PEP 647 (type guards) -- final call for comments

2021-02-14 Thread Gregory P. Smith
(there's a small TL;DR towards the end of my reply if you want to read in reverse to follow my thought process from possible conclusions to how i got there - please don't reply without reading the whole thing first) *TL;DR of my TL;DR* - Not conveying bool-ness directly in the return annotation is

[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-21 Thread Gregory P. Smith
On Sun, Feb 21, 2021 at 10:15 AM Christian Heimes wrote: > On 21/02/2021 13.47, glaub...@debian.org wrote: > > Rust doesn't keep any user from building Rust for Tier 2 or Tier 3 > platforms. There is no separate configure guard. All platforms that Rust > can build for, are always enabled by defau

[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-21 Thread Gregory P. Smith
On Sun, Feb 21, 2021 at 12:03 PM Michał Górny wrote: > On Sun, 2021-02-21 at 13:35 +0100, Christian Heimes wrote: > > On 21/02/2021 13.13, Victor Stinner wrote: > > > Hi, > > > > > > I propose to actively remove support for *legacy* platforms and > > > architectures which are not supported by Pyt

[Python-Dev] Re: New public C API functions must not steal references or return borrowed references

2021-03-25 Thread Gregory P. Smith
On Thu, Mar 25, 2021 at 11:58 AM Mark Shannon wrote: > Hi Victor, > > I'm with you 100% on not returning borrowed references, doing so is just > plain dangerous. > > However, is a blanket ban on stealing references the right thing? > > Maybe the problem is the term "stealing". > The caller is tra

[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-03 Thread Gregory P. Smith
On Sat, Apr 3, 2021 at 7:49 PM Terry Reedy wrote: > On 4/3/2021 7:15 PM, Miro Hrončok wrote: > > On 03. 04. 21 21:44, Łukasz Langa wrote: > >> The memory layout of PyThreadState was unintentionally changed in the > >> recent 3.9.3 bugfix release. This leads to crashes on 32-bit systems > >> when

[Python-Dev] Re: Keeping Python a Duck Typed Language.

2021-04-23 Thread Gregory P. Smith
When reading this, I wrote most of it early and left a draft to bake Then deleted a ton of it after other people replied. I'm conscious that my terminology might be all over the map. Keep that in mind before hitting reply. It'll take me a while to digest and pedantically use Luciano's terms,

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Gregory P. Smith
On Fri, May 7, 2021 at 3:01 PM Larry Hastings wrote: > On 5/7/21 2:45 PM, Pablo Galindo Salgado wrote: > > Given that column numbers are not very big compared with line numbers, we > plan to store these as unsigned chars > or unsigned shorts. We ran some experiments over the standard library and

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Gregory P. Smith
On Fri, May 7, 2021 at 2:50 PM Pablo Galindo Salgado wrote: > Hi there, > > We are preparing a PEP and we would like to start some early discussion > about one of the main aspects of the PEP. > > The work we are preparing is to allow the interpreter to produce more > fine-grained error messages,

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-07 Thread Gregory P. Smith
On Fri, May 7, 2021 at 3:24 PM Pablo Galindo Salgado wrote: > Thanks a lot Gregory for the comments! > > An additional cost to this is things that parse text tracebacks not >> knowing how to handle it and things that log tracebacks >> generating additional output. > > We should provide a way for

[Python-Dev] Re: Speeding up CPython

2021-05-07 Thread Gregory P. Smith
On Fri, May 7, 2021 at 6:51 PM Steven D'Aprano wrote: > On Tue, Oct 20, 2020 at 01:53:34PM +0100, Mark Shannon wrote: > > Hi everyone, > > > > CPython is slow. We all know that, yet little is done to fix it. > > > > I'd like to change that. > > I have a plan to speed up CPython by a factor of fiv

[Python-Dev] Re: a name for the ExceptHandler.type when it is a literal tuple of types

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 8:54 AM Thomas Grainger wrote: > That's this bit: > > ``` > except (A, B): >^^ > ``` > > bpo-43149 currently calls it an "exception group", but that conflicts with > PEP 654 -- Exception Groups and except* > > ``` > >>>> try: >... pass >...

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 11:58 AM Pablo Galindo Salgado wrote: > Hi Brett, > > Just to be clear, .pyo files have not existed for a while: >> https://www.python.org/dev/peps/pep-0488/. > > > Whoops, my bad, I wanted to refer to the pyc files that are generated > with -OO, which have the "opt-2" pref

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
7;t work because it will cause crashes when reading pyc > files produced by the interpreter compiled without the flag. > I don't think the optional existence of column number information needs a different kind of pyc file. Just a flag in a pyc file's header at most. It isn't a new

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 2:09 PM Pablo Galindo Salgado wrote: > > Why not put in it -O instead? Then -O means lose asserts and lose > fine-grained tracebacks, while -OO continues to also > strip out doc strings. > > What if someone wants to keep asserts but do not want the extra data? > exactly m

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-08 Thread Gregory P. Smith
On Sat, May 8, 2021 at 2:40 PM Jonathan Goble wrote: > On Sat, May 8, 2021 at 5:08 PM Pablo Galindo Salgado > wrote: > >> > Why not put in it -O instead? Then -O means lose asserts and lose >> fine-grained tracebacks, while -OO continues to also >> strip out doc strings. >> >> What if someone w

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-09 Thread Gregory P. Smith
On Sun, May 9, 2021 at 9:13 AM Antoine Pitrou wrote: > On Sun, 09 May 2021 02:16:02 - > "Jim J. Jewett" wrote: > > Antoine Pitrou wrote: > > > On Sat, 8 May 2021 02:58:40 + > > > Neil Schemenauer nas-pyt...@arctrix.com wrote: > > > > > > It would be cool if we could mmap the pyc files an

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-11 Thread Gregory P. Smith
On Tue, May 11, 2021 at 3:33 PM Mike Miller wrote: > > On 5/11/21 1:57 AM, Baptiste Carvello wrote: > > Le 11/05/2021 à 09:35, Steven D'Aprano a écrit : > >> On Mon, May 10, 2021 at 09:44:05PM -0400, Terry Reedy wrote: > >> > >>> The vanilla interpreter could be updated to recognize when it is >

[Python-Dev] Re: GDB not breaking at the right place

2021-05-25 Thread Gregory P. Smith
On Tue, May 25, 2021 at 7:49 PM Inada Naoki wrote: > On Tue, May 25, 2021 at 5:38 AM Guido van Rossum wrote: > > > > To the contrary, I think if you want the CI jobs to be faster you should > add the CFLAGS to the configure call used to run the CI jobs. > > > > -Og makes it faster not only CI jo

[Python-Dev] Re: Proposal: declare "unstable APIs"

2021-06-03 Thread Gregory P. Smith
Overall agreement. Your list of ast and code objects and bytecode instructions are things that I'd _hope_ people already consider unstable so declaring them as such just makes sense if we're not doing that already. The ideal way to declare an API as unstable is to constantly change it in a breakin

[Python-Dev] Re: Does anyone use threading debug PYTHONTHREADDEBUG=1 env var? Can I remove it?

2021-07-07 Thread Gregory P. Smith
On Wed, Jul 7, 2021 at 2:28 AM Victor Stinner wrote: > Hi, > > Does anyone use threading debug PYTHONTHREADDEBUG=1 env var on a > Python debug build? If not, can I just remove it? > > -- > > To fix a race condition at thread exit on Linux using the glibc, I > removed calls to pthread_exit() (PyTh

[Python-Dev] Re: Making code object APIs unstable

2021-08-16 Thread Gregory P. Smith
Doing a search of a huge codebase (work), the predominant user of PyCode_New* APIs appears to be checked in Cython generated code (in all sorts of third_party OSS projects). It's in the boilerplate that Cython extensions make use of via it's __Pyx_PyCode_New macro. https://github.com/cython/cython/

[Python-Dev] Re: Making code object APIs unstable

2021-08-17 Thread Gregory P. Smith
thon/Compiler/ExprNodes.py?l=397. I don't see anything obvious mentioning the limited API in that code generator. it'd be best to loop in Cython maintainers for more of an idea of Cython's intents and needs with PyCode_New APIs. I've cc'd cython-de...@python.org. -Greg &g

[Python-Dev] Re: PEP 467 feedback from the Steering Council

2021-08-22 Thread Gregory P. Smith
On Tue, Aug 10, 2021 at 3:48 PM Christopher Barker wrote: > On Tue, Aug 10, 2021 at 3:00 PM wrote: > >> The history of bytes/bytearray is a dual-purpose view. It can be used in >> a string-like way to emulate Python 2 string handling (hence all the usual >> string methods and a repr that displa

[Python-Dev] Re: PEP 667: Consistent views of namespaces

2021-08-23 Thread Gregory P. Smith
Just adding a datapoint, searching our internal codebase at work, I only found two things that I'd consider to be uses of PyEval_GetLocals() outside of CPython itself: https://github.com/earwig/mwparserfromhell/blob/develop/src/mwparserfromhell/parser/ctokenizer/tokenizer.c#L220 https://github.com

[Python-Dev] Re: PEP 654 except* formatting

2021-10-03 Thread Gregory P. Smith
On Sun, Oct 3, 2021 at 10:47 AM Łukasz Langa wrote: > > I know it's a bit late for bikeshedding this thing so if we want to be > conservative and stick to the current syntactical options already defined > in PEP 654, I'm voting Option 2 (given the awkwardness of the *(E1, E2) > example). > +1 o

Re: [Python-Dev] cffi in stdlib

2013-03-01 Thread Gregory P. Smith
On Wed, Feb 27, 2013 at 7:57 AM, Eli Bendersky wrote: > > > I read the cffi docs once again and went through some of the examples. I >> > want to divide this to two topics. >> > >> > One is what you call the "ABI" level. IMHO, it's hands down superior to >> > ctypes. Your readdir demo demonstrat

Re: [Python-Dev] Python Language Summit at PyCon: Agenda

2013-03-02 Thread Gregory P. Smith
On Thu, Feb 28, 2013 at 1:15 AM, Nick Coghlan wrote: > On Thu, Feb 28, 2013 at 1:37 PM, Barry Warsaw wrote: > > On Feb 27, 2013, at 11:33 AM, fwierzbi...@gmail.com wrote: > >>The easy part for Jython is pushing some of our "if is_jython:" stuff > >>into the appropriate spots in CPython's Lib/. >

Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to

2013-03-19 Thread Gregory P. Smith
On Tue, Mar 19, 2013 at 9:44 PM, Michael Foord wrote: > > On 19 Mar 2013, at 17:26, Antoine Pitrou wrote: > > > On Wed, 20 Mar 2013 01:22:58 +0100 (CET) > > michael.foord wrote: > >> http://hg.python.org/cpython/rev/684b75600fa9 > >> changeset: 82811:684b75600fa9 > >> user:Michael Foor

Re: [Python-Dev] Can we triple quoted string as a comment?

2013-03-25 Thread Gregory P. Smith
On Mon, Mar 25, 2013 at 2:16 PM, Victor Stinner wrote: > Hi, > > I just realized that the Python peephole optimizer removes useless > instructions like numbers and strings between other instructions, > without raising an error nor emiting an error. Example: > > $ python -Wd -c 'print "Hello"; "Wor

Re: [Python-Dev] Safely importing zip files with C extensions

2013-03-28 Thread Gregory P. Smith
On Thu, Mar 28, 2013 at 9:09 AM, Brett Cannon wrote: > > > > On Thu, Mar 28, 2013 at 10:44 AM, Thomas Heller wrote: > >> Am 27.03.2013 20:38, schrieb Vinay Sajip: >> >> This quote is here to stop GMane complaining that I'm top-posting. Ignore. >>> >>> I've already posted this to distut

Re: [Python-Dev] The end of 2.7

2013-04-06 Thread Gregory P. Smith
I agree with Benjamin though is it really necessary to do two 2.7 releases a year for the last two years? that's rather rapid (but as the release manager its your call). A few of us (sorry I forgot who all was there though I think Martin was?) had a discussion at PyCon a few weeks ago and seemed

Re: [Python-Dev] The end of 2.7

2013-04-07 Thread Gregory P. Smith
On Sun, Apr 7, 2013 at 6:53 AM, Christian Tismer wrote: > Hi Skip, > > > On 07.04.13 14:10, Skip Montanaro wrote: > > I started writing this last night before the flurry of messages which > arrived overnight. I thought originally, "Oh, Skip, you're being too > harsh." But now I'm not so sure.

Re: [Python-Dev] Fighting the theoretical randomness of "is" on immutables

2013-05-06 Thread Gregory P. Smith
On Mon, May 6, 2013 at 7:50 PM, Terry Jan Reedy wrote: > On 5/6/2013 6:34 PM, Antoine Pitrou wrote: > >> On Mon, 06 May 2013 18:23:02 -0400 >> Terry Jan Reedy wrote: >> >>> >>> 'Item' is necessarily left vague for mutable sequences as bytearrays >>> also store values. The fact that Antoine's exa

Re: [Python-Dev] Issue 11406: adding os.scandir(), a directory iterator returning stat-like info

2013-05-13 Thread Gregory P. Smith
On Sun, May 12, 2013 at 3:04 PM, Ben Hoyt wrote: > > And if we're creating a custom object instead, why return a 2-tuple > > rather than making the entry's name an attribute of the custom object? > > > > To me, that suggests a more reasonable API for os.scandir() might be > > for it to be an iter

<    1   2   3   4   5   6   7   >