[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-15 Thread Victor Stinner
On Tue, Dec 14, 2021 at 7:27 PM Eric Snow wrote: > We have some options: > > * live with the full penalty > * make other changes to reduce the penalty to a more acceptable > threshold than 5% > * eliminate the penalty (e.g. claw back 5% elsewhere) The last time I saw a benchmark on immortal objec

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-15 Thread Victor Stinner
Hi, On Tue, Dec 14, 2021 at 6:13 PM Eric Snow wrote: > I'm still hoping to land a per-interpreter GIL for 3.11. There is > still a decent amount of work to be done but little of it will require > solving any big problems: > > * pull remaining static globals into _PyRuntimeState and PyInterpreter

[Python-Dev] Static types and subinterpreters running in parallel

2021-12-16 Thread Victor Stinner
Hi, One option to solve the https://bugs.python.org/issue40601 "[C API] Hide static types from the limited C API" issue without breaking the backward compatibility is to leave the C API and the stable ABI as they are for the main interpreter (static types), but force subinterpreters running in par

[Python-Dev] Re: "immortal" objects and how they would help per-interpreter GIL

2021-12-16 Thread Victor Stinner
On Thu, Dec 16, 2021 at 2:29 AM Nathaniel Smith wrote: > On Wed, Dec 15, 2021 at 3:07 AM Victor Stinner wrote: > > I wrote https://bugs.python.org/issue39511 and > > https://github.com/python/cpython/pull/18301 to have per-interpreter > > None, True and False singletons.

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

2021-12-16 Thread Victor Stinner
On Thu, Dec 16, 2021 at 12:00 AM Guido van Rossum wrote: > Sam's approach is to use the lower bit of the ob_refcnt field to indicate > immortal objects. This would not work given the stable ABI (which has macros > that directly increment and decrement the ob_refcnt field). (...) we can test > f

[Python-Dev] Re: [python-committers] Re: Python 3.11.0a4 is blocked

2022-01-06 Thread Victor Stinner
On Thu, Jan 6, 2022 at 12:33 PM Pablo Galindo Salgado wrote: > * https://bugs.python.org/issue46006 > > Victor made a revert of his PR but unfortunately, we cannot easily backport > it to 3.10 as it affects the ABI. It affects the interpreter state structure > that although is not on the stable

[Python-Dev] Re: PEP 679 – Allow parentheses in assert statements

2022-01-10 Thread Victor Stinner
Does someone know if linters like pylint or pylakes current warn on "assert(test, msg)" statements? If a project use such assertions which are always true, they can start failing wit the PEP 679, right? Hopefully, the fix is easy: removing the parenthesis give the same behavior on old and new Pyth

[Python-Dev] Re: PEP 679 – Allow parentheses in assert statements

2022-01-10 Thread Victor Stinner
On Mon, Jan 10, 2022 at 4:04 PM Guido van Rossum wrote: > Note that Pablo explicitly requested all feedback to Discourse. Oooops sorry, I missed that part of his email! Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2022-01-11 Thread Victor Stinner
On Tue, Dec 7, 2021 at 7:55 PM Sebastian Berg wrote: > One thing we once did in NumPy (for a runtime problem), was to > intentionally break everyone at pre-release/dev time to point out what > code needed fixing. Then flip the switch back at release time as to > not break production. > After a lo

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2022-01-11 Thread Victor Stinner
possible. At the same time, HPy is sufficently removed to be a good "C > extension API" (as opposed to a stable subset of the CPython implementation > API) > that does not leak implementation details. To ensure this latter property is > why > we try to develop everything in pa

[Python-Dev] PEP 674 – Disallow using macros as l-value (version 2)

2022-01-18 Thread Victor Stinner
ine: https://python.github.io/peps/ep-0674/ And you can find the plain text below. Victor PEP: 674 Title: Disallow using macros as l-value Author: Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-Nov-2021 Python-Version: 3.11 Abstract Incompat

[Python-Dev] Re: Discussion about crash tolerance feature for gdbm module

2022-01-18 Thread Victor Stinner
Hi Dong-hee, Can you please show me a short full example opening a database with automatic snapshop recovery (with your proposed API)? Do you ask us our opinion on the Python API that you propose? Or if the whole feature is worth it? Victor On Tue, Jan 18, 2022 at 2:41 AM Dong-hee Na wrote: >

[Python-Dev] Re: Discussion about crash tolerance feature for gdbm module

2022-01-18 Thread Victor Stinner
How does someone know if a database is corrupted? Why is a separated script needed? A single script cannot automatically detect a corrupted database and load the latest snapshot? How is different from simply copying the whole database file? Victor -- Night gathers, and now my watch begins. It s

[Python-Dev] Request to revert unittest and configparser incompatible changes in Python 3.11

2022-01-18 Thread Victor Stinner
ttributeError: module 'configparser' has no attribute 'SafeConfigParser'. Victor Stinner -- in the name of the Python Red Hat / Fedora maintenance team -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-D

[Python-Dev] Re: Request to revert unittest and configparser incompatible changes in Python 3.11

2022-01-18 Thread Victor Stinner
On Tue, Jan 18, 2022 at 5:48 PM Steve Dower wrote: > Discovering during alpha that some packages haven't updated for the > release that hasn't happened yet isn't the end of the world. Reverting > the changes now is probably a bit premature - realistically we can undo > these anytime during beta if

[Python-Dev] Re: Request to revert unittest and configparser incompatible changes in Python 3.11

2022-01-19 Thread Victor Stinner
On Wed, Jan 19, 2022 at 11:58 AM Sebastian Rittau wrote: > > Am 18.01.22 um 22:57 schrieb Victor Stinner: > > At the end of my first email, I also suggest thinking about > > incompatible changes differently, try to make affected projects > > compatible in advance. The pro

[Python-Dev] Re: PEP 674 – Disallow using macros as l-value (version 2)

2022-01-26 Thread Victor Stinner
Hi, My PEP 674 proposed to change PyDescr_TYPE() and PyDescr_NAME() macros. This change breaks M2Crypto and mecab-python3 projects in code generated by SWIG. I tried two solutions to prevent SWIG accessing PyDescrObject members directly: https://bugs.python.org/issue46538 At the end, IMO it's too

[Python-Dev] Re: PEP 674 – Disallow using macros as l-value (version 2)

2022-01-26 Thread Victor Stinner
On Thu, Jan 27, 2022 at 1:14 AM Larry Hastings wrote: > Just so I understand: is this effectively permanent? My first thought was, > we get SWIG to change its code generation, and in a couple of years we can > follow up and make this change. But perhaps there's so much SWIG code out > there a

[Python-Dev] Python no longer leaks memory at exit

2022-01-27 Thread Victor Stinner
ode, _overlapped, _curses_panel, termios, _sha256, scproxy, cmath, _lsprof, unicodedata) * Nick Coghlan (PEP 489, PEP 573) * Paulo Henrique Silva (modules: time, operator, _functools) * Petr Viktorin (PEP 489, PEP 573, PEP 630) * Stefan Behnel (PEP 489) * Victor Stinner (modules _string, mashal,_imp, _warni

[Python-Dev] Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-01-28 Thread Victor Stinner
Hi, There is a reason why I'm bothering C extensions maintainers and Python core developers with my incompatible C API changes since Python 3.8. Let me share my plan with you :-) In 2009 (Python 3.2), Martin v. Löwis did an amazing job with the PEP 384 "Defining a Stable ABI" to provide a "limit

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-01-28 Thread Victor Stinner
Wait, where is the HPy project in that plan? :-) The HPy project (brand new C API) is a good solution for the long term! My concerns about HPy right now is that, in short, CPython has to continue supporting the C API for a few more years, and we cannot evolve CPython before it will become reasonab

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-01-29 Thread Victor Stinner
On Fri, Jan 28, 2022 at 6:28 PM Guido van Rossum wrote: > I think we will get *one* chance in the next decade to get it right. Whether > that's HPy or evolution of the C API I'm not sure. Would you mind to elaborate? Which risk do you expect from switching to HPy and from fixing the C API (intro

[Python-Dev] Re: PEP 674 – Disallow using macros as l-value (version 2)

2022-01-31 Thread Victor Stinner
On Mon, Jan 31, 2022 at 2:31 PM Petr Viktorin wrote: > > PEP: 674 > > Title: Disallow using macros as l-value > > The current PEP is named "Disallow using Py_TYPE() and Py_SIZE() macros > as l-values", which is misleading: it proposes to change 65 macros. Right, I made changes since I posted the

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-01-31 Thread Victor Stinner
On Mon, Jan 31, 2022 at 1:48 PM Petr Viktorin wrote: > (* we could also break the stable ABI, and we could even do it > reasonably safely over a long period of time, but that's a whole > different discussion.) IMO the stable ABI must be change in the long term, it still leaks too many implementat

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-01-31 Thread Victor Stinner
On Mon, Jan 31, 2022 at 4:03 PM Petr Viktorin wrote: > > If we change the stable ABI, I would prefer to fix multiple issues at > > once. Examples: > > > > * No longer return borrowed references (ex: PyDict_GetItem is part of > > the stable ABI) and no longer steal references (ex: > > PyModule_AddO

[Python-Dev] Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-01 Thread Victor Stinner
Hi, It became more and more common that a C API incompatible change introduced in Python breaks Cython and all Python projects using Cython (ex: numpy). Hopefully, usually only some projects using Cython are broken, not all of them. Some of you may remind the PEP 590 (vectorcall) implementation w

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-01 Thread Victor Stinner
On Tue, Feb 1, 2022 at 4:42 PM Christian Heimes wrote: > I would prefer if we can get Cython and all the other code generator and > bindings library off the unstable C-API. They should use the limited API > instead. If they require any C-APIs outside the limited API, then we > should investigate a

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-01 Thread Victor Stinner
On Tue, Feb 1, 2022 at 5:37 PM Irit Katriel wrote: > _PyErr_StackItem is not part of the C API, it's an internal struct that > cython accesses directly. numpy currently fails on building Cython __Pyx_PyErr_GetTopmostException() function which access tstate->exc_info->exc_type, so it's about the

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-02 Thread Victor Stinner
I wish that there would be a 3rd option: ship C code generated by Cython *but* run Cython if this C code "looks" outdated, for example if building the C code fails with a compiler error. So people using stable Python versions like Python 3.10 would not need Cython, but people testing the "next Pyt

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-02 Thread Victor Stinner
On Wed, Feb 2, 2022 at 9:25 AM wrote: > Guido van Rossum wrote: > > My question for you is if you're willing to write up a list of things in > > CPython that you depend on. Or is this just something you're not willing to > > commit to? It would be nice to know which it is, just so the CPython team

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-02 Thread Victor Stinner
On Wed, Feb 2, 2022 at 3:54 PM Stefan Behnel wrote: > > So people using stable Python versions like Python 3.10 would not need > > Cython, but people testing the "next Python" (Python 3.11) would not > > have to manually removed generated C code. > > That sounds like an environment variable might

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Victor Stinner
On Wed, Feb 2, 2022 at 11:49 PM Eric Snow wrote: > FTR, here is the (private/internal) C-API affected by getting rid of > _Py_IDENTIFIER(): > > * 21 C-API functions with `_Py_Identifer` parameters - would be dropped >+ _PyUnicode_FromId() >+ _PyUnicode_EqualToASCIIId() >+ _PyObject_Cal

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Victor Stinner
On Wed, Feb 2, 2022 at 11:43 PM Eric Snow wrote: > My plan is to replace our use of _Py_IDENTIFIER() with statically > initialized string objects (as fields under _PyRuntimeState). That > involves the following: > > * add a PyUnicodeObject field (not a pointer) to _PyRuntimeState for > each strin

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Victor Stinner
By the way, Argument Clinic now produces way faster calling conventions than hand-written METH_VARARGS with PyArg_ParseTuple(). It would be make this tool available to 3rd party projects. Either extract it and put it on PyPI, but it means that Python will need to Python and pip install something t

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-02-03 Thread Victor Stinner
Hi Guido, My "north star", as you say, is the HPy "design" (not the actual HPy API). I would like to convert PyObject* to opaque handles: dereferencing a PyObject* pointer would simply fail with a compiler error. I'm working bottom-to-top: prepare PyObject and PyVarObject to become opaque, *and*

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-03 Thread Victor Stinner
bject* obj, PyObject* attr_name) { Oh, that's not "_PyObject_LookupSpecial()", it doesn't use the _Py_Identifier type: static CYTHON_INLINE PyObject* __Pyx_PyObject_LookupSpecial(PyObject* obj, PyObject* attr_name) { ... } Victor On Thu, Feb 3, 2022 at 7:27 PM wrote: > > Vi

[Python-Dev] Re: Moving away from _Py_IDENTIFIER().

2022-02-04 Thread Victor Stinner
On Thu, Feb 3, 2022 at 11:49 PM Eric Snow wrote: > I suppose I'd like to know what the value of _Py_IDENTIFIER() is for > 3rd party modules. _Py_IDENTIFIER() API is simple and short, Python creates the Python string object automatically, *and* clears this object at exit. It's convenient. On the

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-02-04 Thread Victor Stinner
On Fri, Feb 4, 2022 at 12:52 AM Eric V. Smith wrote: > > On 2/3/2022 12:15 PM, Victor Stinner wrote: > > > > IMO if PyObject* becomes a handle, the migration to the HPy API should > > be much easier. > > It seems to me that moving PyObject* to be a handle leaves yo

[Python-Dev] It's now time to deprecate the stdlib urllib module

2022-02-06 Thread Victor Stinner
Hi, I propose to deprecate the urllib module in Python 3.11. It would emit a DeprecationWarning which warn users, so users should consider better alternatives like urllib3 or httpx: well known modules, better maintained, more secure, support HTTP/2 (httpx), etc. I don't propose to schedule its re

[Python-Dev] Re: I want to contribute to Python.

2022-02-06 Thread Victor Stinner
On Sun, Feb 6, 2022 at 3:33 PM Ezekiel Adetoro wrote: > Hello, > My name is Ezekiel, and it is my desire to start contributing to Python, be > part of the core development of Python. I have forked the CPython and cloned > it. What is the next step I need to do? Welcome Ezekiel! I suggest you to

[Python-Dev] Re: It's now time to deprecate the stdlib urllib module

2022-02-06 Thread Victor Stinner
On Sun, Feb 6, 2022 at 3:48 PM Damian Shaw wrote: > > Pip vendors requests for network calls: > https://github.com/pypa/pip/tree/main/src/pip/_vendor/requests > > But still does depend on functions from urllib.parse and urllib.request in > many places: > https://github.com/pypa/pip/blob/main/sr

[Python-Dev] Re: It's now time to deprecate the stdlib urllib module

2022-02-06 Thread Victor Stinner
On Sun, Feb 6, 2022 at 3:35 PM Paul Moore wrote: > urllib.request may not be "best practice", but it's still extremely > useful for simple situations, and urllib.parse is useful for basic > handling of URLs.Yes, the more complex aspects of urllib are better > handled by external packages, but that

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-02-07 Thread Victor Stinner
On Mon, Feb 7, 2022 at 2:08 PM Petr Viktorin wrote: > Basically, instead of "We'll remove this API now because it prevents > moving to a hypothetical moving garbage collector", it should be "Here > is a moving garbage collector that speeds Python up by 30%, but to add > it we need to remove these

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-02-07 Thread Victor Stinner
On Mon, Feb 7, 2022 at 2:26 PM Victor Stinner wrote: > CPython is also affected by these issues, but the benefits of PEP 674 > (alone) are too indirect, so I chose to avoid mentioning CPython > issues directly, to avoid confusion. A concrete example of problem caused by exposing stru

[Python-Dev] Re: PEP 674 – Disallow using macros as l-value (version 2)

2022-02-07 Thread Victor Stinner
On Mon, Jan 31, 2022 at 3:57 PM Petr Viktorin wrote: > Remove the point from the Abstract? > (...) > I think you should make it clear that the percentage was taken after > fixing many of the projects. Thanks for your review. I pushed a change to take your remarks in account: https://github.com/p

[Python-Dev] Replace debug runtime checks in release mode with assertions in debug mode

2022-02-07 Thread Victor Stinner
Hi, I would like to replace runtime checks on C function arguments with assertions. For example, replace: PyObject * PyDict_GetItemWithError(PyObject *op, PyObject *key) { if (!PyDict_Check(op)) { PyErr_BadInternalCall(); return NULL; } ... } with: PyObject * PyDict_

[Python-Dev] Re: Replace debug runtime checks in release mode with assertions in debug mode

2022-02-07 Thread Victor Stinner
On Mon, Feb 7, 2022 at 5:38 PM Guido van Rossum wrote: > ISTM this is better discussed on a case-by-case basis than as a blanket > policy change. (The latter could end up causing a flood of trivial PRs from > wannabe-contributors who found and fix yet another violation of the policy, > which is

[Python-Dev] Re: Replace debug runtime checks in release mode with assertions in debug mode

2022-02-07 Thread Victor Stinner
On Mon, Feb 7, 2022 at 5:48 PM Guido van Rossum wrote: > So you're proposing to completely get rid of those three? I don't propose to remove them, but only call them if Python is built in debug mode. Or remove them from the release build, unless ./configure --with-assertions is used. > And you'

[Python-Dev] Require a C compiler supporting C99 to build Python 3.11

2022-02-07 Thread Victor Stinner
Hi, I made a change to require C99 "NAN" constant and I'm was asked to update the PEP 7 to clarify the C subset is needed to build Python 3.11. Python 3.6 requires a subset of the C99 standard to build defined by the PEP 7: https://www.python.org/dev/peps/pep-0007/ I modified Python 3.11 to req

[Python-Dev] Re: Should we require IEEE 754 floating-point for CPython?

2022-02-07 Thread Victor Stinner
Hi Mark, Aha, good, you posted an email to python-dev, good :-) Last days, I was trying to collect more data about this topic, especially find platforms which *don't* support IEEE 754, before posting to python-dev. Nowadays, outside museums, it's hard to find computers which don't implement IEEE

[Python-Dev] Re: Should we require IEEE 754 floating-point for CPython?

2022-02-07 Thread Victor Stinner
On Tue, Feb 8, 2022 at 12:19 AM Brett Cannon wrote: > Do we have a buildbot that has a CPU or OS that can't handle IEEE-754? What > are the chances we will get one? If the answers are "none" and "slim", then > it seems reasonable to require NaN and IEEE-754. All CPU architectures of current bui

[Python-Dev] Re: Replace debug runtime checks in release mode with assertions in debug mode

2022-02-08 Thread Victor Stinner
On Tue, Feb 8, 2022 at 9:49 AM Serhiy Storchaka wrote: > > 07.02.22 17:55, Victor Stinner пише: > > I'm asking to replace runtime checks with assertions when the C API is > > "obviously" misused: replace PyErr_BadInternalCall(), > > _Py_CheckFunction

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

2022-02-08 Thread Victor Stinner
On Tue, Feb 8, 2022 at 2:02 PM Steve Dower wrote: > All the C99 library is supposedly supported, but there are (big?) gaps > in the compiler support. Some Visual Studio 2019 updates on the Microsoft blog. March 2020: C99 _Pragma https://devblogs.microsoft.com/cppblog/announcing-full-support-for-

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

2022-02-08 Thread Victor Stinner
Hi Petr, Thanks for the SC review, it's very helpful! I know that it's a big PEP :-) On Tue, Feb 8, 2022 at 11:33 AM Petr Viktorin wrote: > *All other things being equal, static inline functions are better than > macros.* > Specifically, inline static functions should be preferred over > functio

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

2022-02-09 Thread Victor Stinner
On Wed, Feb 9, 2022 at 1:04 PM Petr Viktorin wrote: > > Right now, a large number of macros cast their argument to a type. A > > few examples: > > > > #define PyObject_TypeCheck(ob, type) > > PyObject_TypeCheck(_PyObject_CAST(ob), type) > > #define PyTuple_GET_ITEM(op, i) (_PyTuple_CAST(op)->ob_it

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-09 Thread Victor Stinner
Hi, It's already possible to call PyObject_CallMethod(code, "co_positions", NULL) and then use the iterator in C. Is there an use case where performance of reading co_positions is critical? If not, there is no need to add a specialized function. Victor On Wed, Feb 9, 2022 at 5:23 PM Petr Viktori

[Python-Dev] Re: Please update Cython *before* introcuding C API incompatible changes in Python

2022-02-09 Thread Victor Stinner
On Wed, Feb 9, 2022 at 5:48 PM Pablo Galindo Salgado wrote: > We consider the representation of co_postions private, so we don't want (for > now) to ad > getters/setters. If you want to get the position of a instruction, you can > use PyCode_Addr2Location The code.co_positions() method is acces

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

2022-02-10 Thread Victor Stinner
On Thu, Feb 10, 2022 at 10:58 AM Petr Viktorin wrote: > > On 09. 02. 22 21:41, Gregory P. Smith wrote: > > > > On Wed, Feb 9, 2022 at 8:54 AM Victor Stinner > Perhaps use a hybrid approach > > when feasible similar to: > >#define PyUnicode_CHECK_INTERNED(

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

2022-02-10 Thread Victor Stinner
On Thu, Feb 10, 2022 at 10:28 AM Petr Viktorin wrote: > Ah, I've also wanted anonymous unions in the past! > There's a little problem in that they're not valid in C++, so we can't > have them in public headers. > > We'll need to mention C++ if we update the standard. IMO we only have to care abou

[Python-Dev] Move the pythoncapi_compat project under the GitHub Python or PSF organization?

2022-02-10 Thread Victor Stinner
Hi, Would it make sense to move the pythoncapi_compat project under the GitHub Python or PSF organization to make it more "official" and a little bit more sustainable? "The pythoncapi_compat project can be used to write a C extension supporting a wide range of Python versions with a single code b

[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?

2022-02-10 Thread Victor Stinner
I created a poll on Discourse: https://discuss.python.org/t/move-the-pythoncapi-compat-project-under-the-github-python-or-psf-organization/13651 It will be closed automatically in 10 days. Victor ___ Python-Dev mailing list -- python-dev@python.org To u

[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?

2022-02-11 Thread Victor Stinner
On Fri, Feb 11, 2022 at 12:06 PM Petr Viktorin wrote: > Or will this send a message that core devs should co-maintain the project? > I personally wouldn't want to maintain it, but it it looks like there > are at least 3 maintainers who do. I think that Neal provided a better answer than me :-) It

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

2022-02-11 Thread Victor Stinner
IMO we need to distinguish the public C API which should be as much compatible as possible, target the oldest C standard, and the Python internals can require a more recent C standard. For example, today maybe it's reasonable to requires C99 for Include/ headers (public .h files) and support C11 f

[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?

2022-02-11 Thread Victor Stinner
On Fri, Feb 11, 2022 at 3:14 PM Petr Viktorin wrote: > Sounds reasonable, but... > > The implication of endorsing code like this is that *we cannot change > private API even in patch releases*, which I don't think is documented > anywhere, and might be a bit controversial. > (I'm still planning to

[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?

2022-02-21 Thread Victor Stinner
Results of the poll (which was open for 10 days): * Move pythoncapi_compat: 19 votes (90%) * Don't move pythoncapi_compat: 2 votes (10%) Victor On Fri, Feb 11, 2022 at 12:16 AM Victor Stinner wrote: > > I created a poll on Discourse: > https://discuss.python.org/t/move-the-pyt

[Python-Dev] Re: Steering Council reply to PEP 670 -- Convert macros to functions in the Python C API

2022-02-21 Thread Victor Stinner
a silver bullet ;-) Victor On Mon, Feb 14, 2022 at 11:29 AM Antoine Pitrou wrote: > > On Wed, 9 Feb 2022 17:49:19 +0100 > Victor Stinner wrote: > > On Wed, Feb 9, 2022 at 1:04 PM Petr Viktorin wrote: > > > > Right now, a large number of macros cast their

[Python-Dev] Version 2 of PEP 670 – Convert macros to functions in the Python C API

2022-02-22 Thread Victor Stinner
Hi, Since Erlend and me posted PEP 670 on python-dev last October, we took all feedback (python-dev and Steering Council) in account to clarify the intent of the PEP and to reduce its scope (remove *any* risk of backward compatibility). PEP 670: https://python.github.io/peps/pep-0670/ Changes si

[Python-Dev] Re: SC reply to PEP 674 -- Disallow using macros as l-values

2022-02-22 Thread Victor Stinner
Hi, Thanks for looking into my PEP 674! I don't understand well why Py_SIZE() cannot be changed until Py_SET_SIZE() is available on all supported Python versions (so affected projects don't have to add a compatibility code), whereas it's ok to require a compatibility code to keep support for Pyth

[Python-Dev] Re: SC reply to PEP 674 -- Disallow using macros as l-values

2022-02-22 Thread Victor Stinner
On Tue, Feb 22, 2022 at 1:54 PM Petr Viktorin wrote: > First, add notices to any documentation that using the macro as a > l-value is a deprecated, CPython-specific detail. It should be clear > that the only reason this usage is allowed is for backwards > compatibility, and that alternate implemen

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

2022-02-23 Thread Victor Stinner
Hi, I updated my PEP 7 PR to use C99 in the public C API and "a subset of" C11 in Python internals: "Python 3.11 and newer versions use C99 in the public C API and use a subset of C11 in Python internals. The public C API should be compatible with C++. The C11 subset are features supported by GCC

[Python-Dev] Re: Version 2 of PEP 670 – Convert macros to functions in the Python C API

2022-02-23 Thread Victor Stinner
On Wed, Feb 23, 2022 at 7:11 PM Petr Viktorin wrote: > In the PEP, the "Performance and inlining" section seems unnecessary. It > talks about attributes that aren't used in the implementation. Or are > they? How does the section relate to the rest of the PEP? > The "Benchmark comparing macros and

[Python-Dev] Re: Version 2 of PEP 670 – Convert macros to functions in the Python C API

2022-02-23 Thread Victor Stinner
On Wed, Feb 23, 2022 at 7:11 PM Petr Viktorin wrote: > I did realize there's one more issue when converting macros or static > inline functions to regular functions. > Regular functions' bodies aren't guarded by limited API #ifdefs, so if > they are part of the limited API it's easy to forget to t

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

2022-02-24 Thread Victor Stinner
Ok, let me try something simpler: "Python 3.11 and newer versions use C11 without optional features. The public C API should be compatible with C++." https://github.com/python/peps/pull/2309/files Victor ___ Python-Dev mailing list -- python-dev@python.

[Python-Dev] Re: Require a C compiler supporting C99 to build Python 3.11

2022-02-24 Thread Victor Stinner
On Thu, Feb 24, 2022 at 11:10 PM Barry wrote: > > "Python 3.11 and newer versions use C11 without optional features. The > > public C API should be compatible with C++." > > https://github.com/python/peps/pull/2309/files > > Should is often read as meaning optional when writing specs. > Can you s

[Python-Dev] Re: Should we require IEEE 754 floating-point for CPython?

2022-02-24 Thread Victor Stinner
The consensus is to require IEEE 754 to build CPython, but not require it in the Python language specification. Updates (changed merged in bpo-46656): * Building Python 3.11 now requires a C11 compiler without optional C11 features. I wrote it in What's New in Python 3.11 and the PEP 7. * Buildin

[Python-Dev] Re: Improvements to the sys.path initialization documentation

2022-03-03 Thread Victor Stinner
Hi, I tried to document inputs and outputs of Modules/getpath.py in the C API Initialization API: https://docs.python.org/dev/c-api/init_config.html#python-path-configuration I don't know if it would be possible to merge most of these information at a single place to avoid oudated documentation.

[Python-Dev] Re: [python-committers] Re: Python 3.11.0a6 is blocked

2022-03-03 Thread Victor Stinner
By the way, AMD64 Arch Linux Usan 3.x started failing because I enabled more tests on this buildbot yesterday. Previously, "test_faulthandler test_hashlib test_concurrent_futures test_ctypes" were simply skipped on this UBSAN buildbot. I'm working on fixing the 3 failing tests: test_ctypes, test_

[Python-Dev] Re: Improvements to the sys.path initialization documentation

2022-03-04 Thread Victor Stinner
Unrelated to the doc, now that getpath has been rewritten in pure Python, thanks to Steve Dower!, it would be nice to move the last bits of the sys.path initialization from the site module to the getpath module. It's unpleasant to have a different sys.path depending if the site module is loaded or

[Python-Dev] Re: Improvements to the sys.path initialization documentation

2022-03-07 Thread Victor Stinner
On Fri, Mar 4, 2022 at 1:37 PM Eryk Sun wrote: > I don't understand. The site packages directories, including virtual > environments, are a site extension. I propose to change that. Move all code related to sys.path from the site module to the Modules/getpath.py module. Victor -- Night gathers,

[Python-Dev] Re: Defining tiered platform support

2022-03-11 Thread Victor Stinner
It would be great to have the list of supported platforms per Python version! Maybe supporting new platforms and dropping support for a platform should be document in What's New in Python x.y. GCC does that for example. It also *deprecates* support for some platforms. Example: https://gcc.gnu.org/

[Python-Dev] SC accepted PEP 594: Removing dead batteries from the standard library

2022-03-14 Thread Victor Stinner
Hi, Oh, the Steering Council accepted PEP 594 "Removing dead batteries from the standard library"! I just saw the announcement on Discourse. Congratulations Christian and Brett! This PEP, first proposed in 2019, wasn't an easy one. https://peps.python.org/pep-0594/ Gregory P. Smith's message on

[Python-Dev] C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-03-22 Thread Victor Stinner
Hi, I proposed two PRs to move the private C API (Include/cpython/) of PEP 523 "Adding a frame evaluation API to CPython" to the internal C API (Include/internals/): * https://github.com/python/cpython/pull/32052 * https://github.com/python/cpython/pull/32054 API: * _PyFrameEvalFunction type *

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-03-22 Thread Victor Stinner
On Tue, Mar 22, 2022 at 7:33 PM Steve Dower wrote: > After a normal deprecation period, yes? There is no backward compatibility warranty and no deprecation process for private APIs. Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubsc

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-03-30 Thread Victor Stinner
On Wed, Mar 30, 2022 at 2:26 AM Steve Dower wrote: > Right now, the API is allowed/expected to change between 3.x releases > (which normally we don't allow without a deprecation period) but it > still has to remain compatible within a single 3.x release. Making it > fully internal *without adding

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-03-30 Thread Victor Stinner
The internal C API can be used on purpose. But there is no backward compatibility warranty and it can change anytime. In practice, usually it only changes in 3.x.0 releases. For example, these private C API changed in Python 3.9 and Python 3.11 (see my first email in the other PEP 523 thread). To

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-03-31 Thread Victor Stinner
On Wed, Mar 30, 2022 at 9:26 PM Sebastian Berg wrote: > That is fair, although there are documented underscored ones: > https://docs.python.org/3/search.html?q=_Py > > I suppose that means all bets are off _unless_ it is documented or > later adopted as stable API (e.g. `_PyObject_Vectorcall`)

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-04-01 Thread Victor Stinner
On Mon, Mar 28, 2022 at 1:44 PM Petr Viktorin wrote: > Perhaps we need a new "tier" of C API for debuggers -- API that's > guaranteed stable for a major release, and if it's changed it should > always break with compile errors (e.g. the function gets a new > argument), rather than silently change

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-04-01 Thread Victor Stinner
On Wed, Mar 30, 2022 at 5:42 PM Guido van Rossum wrote: > In the not so distant past I have proposed to introduce a new category, > "Unstable APIs". These are public but are not guaranteed to be backwards > compatible in feature releases (though I feel they should remain so in bugfix > releases

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-04-01 Thread Victor Stinner
Hi, Update on this issue: I merged my 2 PRs. https://bugs.python.org/issue46850 The following APIs have been moved to the internal C API: - _PyFrameEvalFunction type - _PyInterpreterState_GetEvalFrameFunc() - _PyInterpreterState_SetEvalFrameFunc() - _PyEval_EvalFrameDefault() If you use any of

[Python-Dev] code.replace() and Python 3.11 exception table

2022-04-01 Thread Victor Stinner
("Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API") On Fri, Apr 1, 2022 at 11:01 AM Chris Angelico wrote: > > On Fri, 1 Apr 2022 at 19:51, Victor Stinner wrote: > > In Python, sadly the types.CodeType ty

[Python-Dev] Re: code.replace() and Python 3.11 exception table

2022-04-01 Thread Victor Stinner
I created https://bugs.python.org/issue47185 to discuss this issue: either recompute automatically co_exceptiontable, or at least document the change. Victor On Fri, Apr 1, 2022 at 11:21 AM Victor Stinner wrote: > > ("Re: C API: Move PEP 523 "Adding a frame evaluation API to CP

[Python-Dev] Re: Make HAMT available to python script

2022-04-01 Thread Victor Stinner
Hi, In 2019, Yury Selivanov, who added HAMT and contextvars to Python, wrote PEP 603 "Adding a frozenmap type to collections": https://peps.python.org/pep-0603/ Sadly, the PEP was stuck in discussions: * https://discuss.python.org/t/pep-603-adding-a-frozenmap-type-to-collections/2318 * https://

[Python-Dev] Re: Make HAMT available to python script

2022-04-03 Thread Victor Stinner
PEP 603 adds a frozenmap type to collections, implemented as HAMT. For a read-only *dict*, I proposed PEP 416 "Add a frozendict builtin type" in 2012. It was rejected. https://peps.python.org/pep-0416/ The outcome of this PEP was the addition of the types.MappingProxy type (previously, it already

[Python-Dev] Re: Slowly bend the C API towards the limited API to get a stable ABI for everyone

2022-04-05 Thread Victor Stinner
IMO it would be better to keep the HPy design as the long term goal: * Refer to Python objects with opaque handles * All structures are opaque (with a few exceptions, like PyType_Spec) It will likely take multiple iterations (Python releases) to reach this goal, and incompatible C API changes may

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-04-05 Thread Victor Stinner
On Sun, Apr 3, 2022 at 3:29 PM Nick Coghlan wrote: > The changes you've made have been excellent, and the existing 3 categories > (stable public ABI, stable public API, unstable internal API) cover the vast > majority of cases. > > The final case that isn't quite covered yet is to offer a "semi-

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-04-05 Thread Victor Stinner
Hi, Steve, Petr: sorry if you feel that I didn't take your feedback in account, it's not case. Thanks for your valuable feedback. It seems like there was some misunderstanding. On Tue, Apr 5, 2022 at 2:49 AM Gregory P. Smith wrote: > Thanks for bringing this up on python-dev, Victor. That was g

[Python-Dev] Re: [python-committers] Re: [IMPORTANT] Preparations for 3.11.0 beta 1

2022-04-07 Thread Victor Stinner
IMO adding locale.getencoding() to Python 3.11 is not controversial and is useful even if PEP 686 is rejected. This function was discussed for 1 year (bpo-43510, bpo-43552, bpo-43557, bpo-47000) and there is an agreement that there is a need for this function. > Making `open(path, encoding="locale

[Python-Dev] Re: [python-committers] Re: [IMPORTANT] Preparations for 3.11.0 beta 1

2022-04-07 Thread Victor Stinner
On Thu, Apr 7, 2022 at 5:35 AM Inada Naoki wrote: > I just submitted the PEP 686 to the SC. > https://github.com/python/steering-council/issues/118 > > In this PEP, I am proposing: > > a. Small improvement for UTF-8 mode in Python 3.11 > b. Make UTF-8 mode default in Python 3.13. It's easier to a

[Python-Dev] About PEPs being discussed on Discourse

2022-04-07 Thread Victor Stinner
Hi, Would it be possible to announce new PEPs on python-dev please? I don't go often to Discourse, like once a month. I don't get any notification by email. I expected new PEPs to be announced on python-dev, but they are not announced here anymore. Is it possible to get Discourse notifications by

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

2022-04-07 Thread Victor Stinner
On Thu, Apr 7, 2022 at 12:02 PM Petr Viktorin wrote: > - This API stays with the regular public API (Include/cpython/), but to > use it you'll need to #define Py_USING_UNSTABLE_API (name up for > bikeshedding). Since there is already something similar called "Py_LIMITED", I suggest dropping "USIN

<    15   16   17   18   19   20   21   22   23   24   >