[Python-Dev] Re: nanosecond stat fields, but not os.path methods ?

2020-12-07 Thread Victor Stinner
Le lun. 7 déc. 2020 à 17:22, David Mertz a écrit : > Are there any filesystems that can actually record a meaningful ns > modification time? I find discussions claiming this: > > EXT4: millisecond precision EXT4 and BTRFS have a resolution of 1 nanosecond. > XFS and EXT3: second precision XFS

[Python-Dev] Re: nanosecond stat fields, but not os.path methods ?

2020-12-07 Thread Victor Stinner
Le lun. 7 déc. 2020 à 19:52, Antoine Pitrou a écrit : > > But also notes that the precision is likely to exceed the accuracy by many > > times on real systems. > > Even if the accuracy is much lower than that, it can be important to > reproduce exact timestamps. Sure, while my initial attempt to

[Python-Dev] Can lib2to3 and 2to3 be deprecated yet?

2020-12-10 Thread Victor Stinner
Hi, I vaguely recall that PEP 617 "New PEG parser for CPython" was supposed to deprecate the lib2to3 module and the 2to3 program. I see that lib2to3/__init__.py contains a PendingDeprecationWarning in Python 3.9. Can it be converted into a regular DeprecationWarning and document the deprecation i

[Python-Dev] Re: Can lib2to3 and 2to3 be deprecated yet?

2020-12-10 Thread Victor Stinner
Le jeu. 10 déc. 2020 à 11:51, Matthias Klose a écrit : > lib2to3 is used by distutils/util, so why deprecate it before distutils? I don't think that it's exclusive. We can deprecate the 2to3 feature in distutils as well. But please don't put a requirement on deprecating distutils to be able to de

[Python-Dev] Re: Enum bug?

2020-12-28 Thread Victor Stinner
IMO it's a feature, not a bug :-) >>> import enum >>> class Foo(enum.Enum): ... A = 1 ... B = 1.0 ... >>> Foo(1) >>> Foo(1.0) >>> Foo.B See: https://docs.python.org/dev/library/enum.html#duplicating-enum-members-and-values "However, two enum members are allowed to have the same value. Gi

[Python-Dev] Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-08 Thread Victor Stinner
Hi, In the Python stdlib, many heap types currently don't "properly" (fully?) implement the GC protocol which can prevent to destroy these types at Python exit. As a side effect, some other Python objects can also remain alive, and so are not destroyed neither. There is an on-going effect to dest

[Python-Dev] Re: why include cpython/*.h, Need this macro ? # error "this header file must not be included directly"

2021-01-11 Thread Victor Stinner
On Sun, Jan 10, 2021 at 9:27 PM Serhiy Storchaka wrote: > Because it will not produce compile-time error when you include that > header files directly. > > The division of these declarations on few files is a deep implementation > detail. Header file "cpython/fileobject.h" is not a part of API and

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-11 Thread Victor Stinner
Hi, There are multiple PEPs covering heap types. The latest one refers to other PEPs: PEP 630 "Isolating Extension Modules" by Petr Viktorin. https://www.python.org/dev/peps/pep-0630/#motivation The use case is to embed multiple Python instances (interpreters) in the same application process, or

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Victor Stinner
On Tue, Jan 12, 2021 at 3:28 PM Petr Viktorin wrote: > If a type is immutable and has no references to heap-allocated objects, > it could stay as a static type. > The issue is that very many types don't fit that. For example, if some > method needs to raise a module-specific exception, that's a re

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Victor Stinner
Since documentation changes are backported to 3.8 and 3.9 stable branches, if we increase the minimum required Sphinx version in master, I would prefer to also increase it in 3.8 and 3.9 branches. I would prefer to not have to check manually if a doc backport PR is still compatible with Sphinx 2 o

[Python-Dev] Re: seg fault in 3.10a4

2021-01-17 Thread Victor Stinner
See https://github.com/python-pillow/Pillow/issues/5193 and https://github.com/python-pillow/Pillow/pull/5194 for a Pillow crash on Python 3.10. A fix is merged in master, but there is no release containing the fix yet. https://github.com/python-pillow/Pillow/commit/543fa2ceb78103c7ae098900c315748a

[Python-Dev] Please explain how to migrate when a function is removed, thanks ;-)

2021-01-19 Thread Victor Stinner
Hi, We are working on upgrading Python from 3.9 to 3.10 in Fedora and we are facing many Python 3.10 incompatible changes. Most changes were planned with a deprecation period, but, as usual, projects are not tested with DeprecationWarning treated as errors, and so tons of packages now fail to buil

[Python-Dev] Re: Please explain how to migrate when a function is removed, thanks ;-)

2021-01-19 Thread Victor Stinner
A PR was proposed to document the removal of the loop parameter: https://github.com/python/cpython/pull/24256 Victor On Tue, Jan 19, 2021 at 1:34 PM Victor Stinner wrote: > > Hi, > > We are working on upgrading Python from 3.9 to 3.10 in Fedora and we > are facing many Python 3.

[Python-Dev] Re: Improve CPython tracing performance

2021-01-20 Thread Victor Stinner
Reply to an old thread. On Sat, Oct 31, 2020 at 8:02 AM Nick Coghlan wrote: > > Debuggers and profilers usually only care of specific frames or > > function calls (ex: 10% of function calls or even a single function > > call in a whole application). The problem is how to make them as > > efficien

[Python-Dev] Re: Please explain how to migrate when a function is removed, thanks ;-)

2021-01-21 Thread Victor Stinner
Currently, we wait until the first user complains, sometimes after a 3.x.0 final release, before starting to document how to port existing code. I agree that it should be done earlier. I suggest that developers who want to introduce an incompatible change think about how to port existing code, esp

[Python-Dev] New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
Hi, I just added a new sys.module_names attribute, list (technically a frozenset) of all stdlib module names: https://bugs.python.org/issue42955 There are multiple use cases: * Group stdlib imports when reformatting a Python file, * Exclude stdlib imports when computing dependencies. * Exclude s

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
On Mon, Jan 25, 2021 at 4:18 PM Antoine Pitrou wrote: > > On Mon, 25 Jan 2021 14:03:22 +0100 > Victor Stinner wrote: > > > > The list is opinionated and defined by its documentation: > > So "the list is opinionated" means there can be false negatives, i.e

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
Hi Ivan, On Mon, Jan 25, 2021 at 4:53 PM Ivan Pozdeev via Python-Dev wrote: > Just _names_? There's a recurring error case when a 3rd-party module > overrides a standard one if it happens to have the same name. If you > filter such a module out, you're shooting yourself in the foot... Overridin

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
() or all available modules. Can we name it instead > sys.stdlib_modules_names to clarify that this is standard library only subset > and not all available modules for the interpreter? > > Thanks, > > On Mon, Jan 25, 2021 at 4:33 PM Victor Stinner wrote: >> >> Hi Iv

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
On Mon, Jan 25, 2021 at 6:39 PM Ethan Furman wrote: > > For packages, only sub-packages are listed, not sub-modules. For > > example, ``concurrent`` package and ``concurrent.futures`` > > sub-package are listed, but not ``concurrent.futures.base`` > > sub-module. > > I'm not sure I

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
On Mon, Jan 25, 2021 at 6:37 PM Chris Jerdonek wrote: > On Mon, Jan 25, 2021 at 7:51 AM Ivan Pozdeev via Python-Dev > wrote: >> >> Just _names_? There's a recurring error case when a 3rd-party module >> overrides a standard one if it happens to have the same name. If you >> filter such a module

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-25 Thread Victor Stinner
On Mon, Jan 25, 2021 at 11:22 PM Ivan Pozdeev via Python-Dev wrote: > That's not possible. > > Stdlib can be arranged any way a user/maintainer wishes (zipped stdlib and > virtual environments are just two examples), so there's no way to tell if the > module's location is "right". > Dowstream ch

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-26 Thread Victor Stinner
On Tue, Jan 26, 2021 at 12:44 AM Steven D'Aprano wrote: > > On Mon, Jan 25, 2021 at 06:17:09PM +0100, Victor Stinner wrote: > > Hi Bernat, > > > > "stdlib_module_names" was my first idea but it looks too long, so I > > chose "module_names".

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-26 Thread Victor Stinner
On Tue, Jan 26, 2021 at 10:04 PM Steve Dower wrote: > > On 1/26/2021 8:32 PM, Steve Holden wrote: > > If the length of the name is any kind of issue, since the stdlib > > only contains modules (and packages), why not just sys.stdlib_names? > > And since the modules can vary between platforms and b

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-27 Thread Victor Stinner
Hi Steven, That makes sense to me: I wrote https://github.com/python/cpython/pull/24353 to exclude sub-package. The change removes 12 sub-packages from sys.stdlib_module_names and len(sys.stdlib_module_names) becomes 300 :-) -"concurrent.futures", -"ctypes.macholib", -"distutils.command", -"emai

[Python-Dev] Re: New sys.module_names attribute in Python 3.10: list of all stdlib modules

2021-01-27 Thread Victor Stinner
On Wed, Jan 27, 2021 at 1:16 AM Steven D'Aprano wrote: > Right. This is (I think) Steve's point: the list is inaccurate, because > the existence of 'winsound' in the stdlib_module_names doesn't mean that > the module 'winsound' exists. This point is addressed by the definition of the list: sys.st

[Python-Dev] Re: Why aren't we allowing the use of C11?

2021-01-29 Thread Victor Stinner
Hi Mark, I tried to use C11 _Thread_local (Thread Local Storage, TLS) only with GCC and clang and I got issues on macOS: https://github.com/python/cpython/pull/23976 My PR uses __thread if _Thread_local is not available. I don't think that MSC (Visual Studio) implements C11 _Thread_local, but it

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-01 Thread Victor Stinner
Hi Inada-san, I followed the discussions on your different PEP and I like overall your latest PEP :-) I have some minor remarks. On Mon, Feb 1, 2021 at 6:55 AM Inada Naoki wrote: > The warning is disabled by default. New ``-X warn_encoding`` > command-line option and ``PYTHONWARNENCODING`` envir

[Python-Dev] Re: PEP 624: Remove Py_UNICODE encoder APIs

2021-02-01 Thread Victor Stinner
On Mon, Feb 1, 2021 at 4:47 PM M.-A. Lemburg wrote: > At the very least, we should have such APIs for going from wchar_t* > to a Python object. > > The alternatives you provide all require creating an intermediate > Python object for this purpose. We cannot optimize all use cases. IMO we should o

[Python-Dev] Re: Why aren't we allowing the use of C11?

2021-02-01 Thread Victor Stinner
On Sat, Jan 30, 2021 at 10:37 AM Antoine Pitrou wrote: > You can hide the access behind a function call. Slightly more costly, > but shouldn't be that expensive on modern machines. Oh sure, on the API side, it can be an "opaque" function call (on the ABI side) or static inline function, depending

[Python-Dev] Re: PEP 624: Remove Py_UNICODE encoder APIs

2021-02-01 Thread Victor Stinner
On Mon, Feb 1, 2021 at 5:39 PM M.-A. Lemburg wrote: > The C code is already there, but it got hidden away in the > Python 3.3 change to new internals. Well, we are not in agreement and it's ok. Your objection is written in the PEP. IMO it's now up to the Steering Council to decide if the overall

[Python-Dev] Re: PEP 624: Remove Py_UNICODE encoder APIs

2021-02-01 Thread Victor Stinner
On Mon, Feb 1, 2021 at 5:58 PM M.-A. Lemburg wrote: > The fix is pretty simple, doesn't add a lot more code and gets > us the symmetry back that I had put into the Unicode C API when > I created this back in 2000. This sounds like a completely different PEP than PEP 624 (which aims to remove code

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-02 Thread Victor Stinner
On Tue, Feb 2, 2021 at 5:40 AM Inada Naoki wrote: > > In Python 3.10, I added _locale._get_locale_encoding() function which > > is exactly what the encoding used by open() when no encoding is > > specified (encoding=None) and when os.device_encoding(fd) returns > > None. See _Py_GetLocaleEncoding(

[Python-Dev] Re: PEP 624: Remove Py_UNICODE encoder APIs

2021-02-02 Thread Victor Stinner
On Tue, Feb 2, 2021 at 11:47 PM Inada Naoki wrote: > So if we support add UTF-16 support to ucs2_utf8_encoder(), it means > we need to add code and maintain only for PyUnicode_EncodeUTF8 (encode > from wchar_t* into char*). > > I don't think it is a good deal. As described in the PEP, encoder APIs

[Python-Dev] Re: Upcoming 3.7.10 and 3.6.13 Security Releases

2021-02-03 Thread Victor Stinner
On Thu, Jan 21, 2021 at 5:09 AM Senthil Kumaran wrote: > And another security issue was brought up to our attention here (today): > https://bugs.python.org/issue42987 This one was already fixed (the issue is closed). Victor -- Night gathers, and now my watch begins. It shall not end until my de

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-09 Thread Victor Stinner
On Sat, Feb 6, 2021 at 3:26 PM Inada Naoki wrote: > I changed my mind. Since there is no plan to change the default > encoding for now, > no need to encourage `encoding="locale"` soon. > > Until users can drop Python 3.9 support, they can use EncodingWarning > only for finding missing `encoding="u

[Python-Dev] Re: Signature discrepancies between documentation and implementation

2021-02-09 Thread Victor Stinner
If there are applications relying on the parameter name, it's better to trust the Python implementation, rather than the documentation. It would be better to make these parameters positional only in the first place, but now it's too late for such backward incompatible change :-( Victor On Tue, F

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-09 Thread Victor Stinner
On Tue, Feb 9, 2021 at 1:31 PM Paul Moore wrote: > If we can't provide a good recommendation > to the user on what to do, we shouldn't be warning them that what they > are currently doing is wrong. The warning can explicitly suggest to use encoding="utf8", it should work in almost all cases. Vic

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-09 Thread Victor Stinner
On Tue, Feb 9, 2021 at 5:51 PM Anders Munch wrote: > Victor Stinner [mailto:vstin...@python.org] wrote: > > The warning can explicitly suggest to use encoding="utf8", it should work > > in almost all cases. > > The warning should also explain how to get backwar

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-10 Thread Victor Stinner
On Tue, Feb 9, 2021 at 9:51 PM Paul Moore wrote: > * Realistically, I'd be surprised if developers actually use such a > tool. If they were likely to do so, they could probably just as easily > locate all the uses of open() in their code, and check that way. So > I'm not sure this proposal is actu

[Python-Dev] Re: Python standardization

2021-02-15 Thread Victor Stinner
See "A formal specification for the (C)Python virtual machine - Python Language Summit 2020" by Mark Shannon: https://pyfound.blogspot.com/2020/04/a-formal-specification-for-cpython.html Victor On Fri, Feb 12, 2021 at 7:40 PM Dan Stromberg wrote: > > > What would it take to create an ANSI, ECMA

[Python-Dev] Re: PEP 597 bikeshedding: envvar / option name.

2021-02-15 Thread Victor Stinner
On Mon, Feb 15, 2021 at 6:31 AM Inada Naoki wrote: > * PYTHONWARNENCODING / warn_ecoding (I love this "ecoding" typo, but it's not my favorite choice ;-)) I dislike "warn_encoding" because it is too generic. An encoding? Which one? Why does anyone want to emit a warning when an encoding is used?

[Python-Dev] Re: Security releases of CPython

2021-02-15 Thread Victor Stinner
Hi Michał, I created https://python-security.readthedocs.io/ website to track known Python vulnerabilities to help me checking if fixes are backported to all supported Python branches. I'm maintaing this list manually, it's far from being complete, and likely outdated. I also created https://gith

[Python-Dev] Re: It is really necessary to check that a object has a Py_TPFLAGS_HEAPTYPE flag?

2021-02-15 Thread Victor Stinner
On Tue, Feb 9, 2021 at 4:53 AM Kazantcev Andrey wrote: > I decided to figure out why it is impossible to change methods of built-in > types and found > ``` > if (!(type->tp_flags & Py_TPFLAGS_HEAPTYPE)) { > PyErr_Format( > PyExc_TypeError, > "can't set attribut

[Python-Dev] Re: PEP 597 bikeshedding: envvar / option name.

2021-02-19 Thread Victor Stinner
Environment variable names with underscore would be more readable, but IMO consistency with all existing names matters more: https://docs.python.org/dev/using/cmdline.html#environment-variables So I prefer PYTHONWARNDEFAULTENCODING env var and sys.flags.warn_default_encoding attribute names. Vict

[Python-Dev] Re: Security releases of CPython

2021-02-20 Thread Victor Stinner
On Thu, Feb 11, 2021 at 9:44 PM Michał Górny wrote: > I feel that vulnerability fixes do not make it to end users fast enough. I think that it's time to put that into perspective with past vulnerabilities. Ok, let me look at the timeline of the discussed vulnerability, ctypes CVE-2021-3177: http

[Python-Dev] Move support of legacy platforms/architectures outside Python

2021-02-21 Thread Victor Stinner
Hi, I propose to actively remove support for *legacy* platforms and architectures which are not supported by Python according to PEP 11 rules: hardware no longer sold and end-of-life operating systems. The removal should be discussed on a case by case basis, but I would like to get an agreement on

[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-22 Thread Victor Stinner
On Sun, Feb 21, 2021 at 8:57 PM Michał Górny wrote: > > The checker serves two purposes: > > > > 1) It gives users an opportunity to provide full PEP 11 support > > (buildbot, engineering time) for a platform. > > Does that mean that if someone offers to run the build bot for a minor > platform an

[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-22 Thread Victor Stinner
On Mon, Feb 22, 2021 at 8:19 AM wrote: > There are zero technical reasons for what you are planning here. Multiple core developers explained how it's a maintenance burden. It has been explained in multiple different ways. > You are inflating a few lines of autoconf into a "platform support", so

[Python-Dev] Re: Deprecate support for mingw - add to PEP 11

2021-02-22 Thread Victor Stinner
pybind11 is a famous C++ extension module for Python. Yes, the Python C API is usable in C++ thanks to extern "C" { ... } being used in headers. Victor On Sun, Feb 21, 2021 at 6:59 PM Dan Stromberg wrote: > > > It looks like CPython remains 100% C, so clang becomes more attractive: > https://st

[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-22 Thread Victor Stinner
On Mon, Feb 22, 2021 at 12:51 PM Ivan Pozdeev via Python-Dev wrote: > IIRC I suggested earlier that buildsbots should be integrated into the PR > workflow in order to make it the contributor's rather than a core > dev's burden to fix any breakages that result from their changes. Some buildbot wo

[Python-Dev] Re: Moving threadstate to thread-local storage.

2021-02-26 Thread Victor Stinner
Sure. The plan is to use __thread when possible ;-) Victor On Thu, Feb 25, 2021 at 4:58 AM 谢俊逸 via Python-Dev wrote: > > On MacOS & iOS, __thread variable is 35% faster than using > pthread_getspecific. > > getSpecific cost: 0.000649 > getTLS cost: 0.000423 > > > - (void)test { double t1 = CFAb

[Python-Dev] Re: Which thing is "Development Mode"

2021-02-26 Thread Victor Stinner
I suggest to assign a color to each development mode to distinguish them. Victor On Fri, Feb 26, 2021 at 7:50 PM Coyot Linden wrote: > > As of 3.7, there is now a python feature called Development Mode: > > https://docs.python.org/3/library/devmode.html#devmode > > Which has a confusingly simila

[Python-Dev] Need help to fix known Python security vulnerabilities

2021-03-08 Thread Victor Stinner
Hi, The Python bug tracker currently has 78 open issues of the type Security. If you are looking for something to do to help the Python project, please go through the list (search for open issues with Type=security at bugs.python.org), discuss the different solutions how to address these vulnerabi

[Python-Dev] Re: CFLAGS_NODIST and -qalias=noansi

2021-03-08 Thread Victor Stinner
On Fri, Mar 5, 2021 at 5:47 PM Michael Felt wrote: > Fyi: using `git log` I have tried to get any clue re: CFLAGS_NODIST > and/or -qalias=noansi - but I do not seem to be skilled enough to find > that information. Try "git blame configure.ac" and search for "noansi". It was introduced in bpo-417

[Python-Dev] Re: Steering Council update for February

2021-03-10 Thread Victor Stinner
Hi, Could we please stick to the point of renaming a Git branch? Today, renaming a branch is easy. The rationale has been given. I don't think any argument is going to make the Steering Concil changing their mind ("the consensus was that we should do that"). If you want to help, please remain at t

[Python-Dev] Re: [Release management] schedule for renaming the default branch

2021-03-11 Thread Victor Stinner
Hi Serhiy, On Thu, Mar 11, 2021 at 8:33 AM Serhiy Storchaka wrote: > I have above 200 feature branches in my local repository. Will renaming > the master branch cause any problems? I don't think that you need to do anything on your machine nor on your open PRs. When I use "git switch -c new_bra

[Python-Dev] Re: Reviving PEP 473

2021-03-13 Thread Victor Stinner
Context: PEP 473 "Adding structured data to built-in exceptions" proposed in 2014. https://www.python.org/dev/peps/pep-0473/ Abstract: "Exceptions like AttributeError, IndexError, KeyError, LookupError, NameError, TypeError, and ValueError do not provide all information required by programmers to

[Python-Dev] Re: [python-committers] Accepting PEP 597 (Add optional EncodingWarning)

2021-03-15 Thread Victor Stinner
Congratulation INADA-san! I'm impressed by your tenacity :-) Last months, I followed your different propositions on discuss.python.org to use UTF-8 by default in Python. It's good to see the first non-controversial part being accepted! I hope that this PEP will help to move towards a world where w

[Python-Dev] Re: Reviving PEP 473

2021-03-16 Thread Victor Stinner
For best performance (reduce time to build an exception object), I would be interested to only format the error message on demand. For example, when str(exception) is called. The problem is the Exception.args attribute. Example: $ ./python Python 3.10.0a6+ >>> exc=AttributeError("%s object has no

[Python-Dev] Re: Non-monotonically increasing line numbers in dis.findlinestarts() output

2021-03-17 Thread Victor Stinner
Hi Skip, I modified co_lnotab format when I worked on the FAT Python optimization project which modified the AST to emit more efficient bytecode. My optimizer moved instructions, the line number could become non-monotonic (..., line 2, line 3, line 2, line 4, ...), and so lnotab (..., +1, -1, +2,

[Python-Dev] Tests now start with only 131 imported modules, instead of 233

2021-03-22 Thread Victor Stinner
Hi, tl;dr Tests of the Python Test Suite now start with 131 imported modules, instead of 233. For example, asyncio, logging, multiprocessing and warnings are no longer imported by default. -- The Python test suite is run by a custom test runner called "libregrtest" (test.libregrtest). It can det

[Python-Dev] Re: Tests now start with only 131 imported modules, instead of 233

2021-03-23 Thread Victor Stinner
Hi Ivan, On Tue, Mar 23, 2021 at 9:04 AM Ivan Pozdeev via Python-Dev wrote: > I didn't quite get what the big effect is. Saving 30 milliseconds? I started to dig into this issue while debugging a random crash on AIX (bpo-40091). A test_threading test using fork randomly crashed on AIX. I discove

[Python-Dev] Re: Tests now start with only 131 imported modules, instead of 233

2021-03-23 Thread Victor Stinner
On Tue, Mar 23, 2021 at 9:04 AM Ivan Pozdeev via Python-Dev wrote: > Also, how is the now-split-off funcionality to be invoked? Does it require > two or more imports now, or it's imported on demand when one > invokes an appropriate test.support entry? By the way, splitting test.support into sub-

[Python-Dev] New public C API functions must not steal references or return borrowed references

2021-03-25 Thread Victor Stinner
Hi, A new Include/README.rst file was just added to document the 3 C API provided by CPython: * Include/: Limited C API * Include/cpython/: CPython implementation details * Include/internal/: The internal API I would like to note that *new* public C API functions must no longer steal references

[Python-Dev] Weird io.OpenWrapper hack to use a function as method

2021-03-31 Thread Victor Stinner
Hi, The io module provides an open() function. It also provides an OpenWrapper which only exists to be able to store open as a method (class or instance method). In the _pyio module, pure Python implementation of the io module, OpenWrapper is implemented as: class OpenWrapper: """Wrapper for

[Python-Dev] Re: Weird io.OpenWrapper hack to use a function as method

2021-03-31 Thread Victor Stinner
ed() --- *Maybe* staticmethod could be modified to become callable? Victor On Wed, Mar 31, 2021 at 2:34 PM Victor Stinner wrote: > > Hi, > > The io module provides an open() function. It also provides an > OpenWrapper which only exists to be able to store open as a method > (clas

[Python-Dev] Re: Weird io.OpenWrapper hack to use a function as method

2021-03-31 Thread Victor Stinner
A friend, Antoine Rozo, wrote a variant of staticmethod which is callable. With this decorator, it works in A, B and C cases: --- class simplefunction: def __init__(self, func): self.func = func def __get__(self, owner, instance): return self.func def __call__(self, *arg

[Python-Dev] Re: Weird io.OpenWrapper hack to use a function as method

2021-03-31 Thread Victor Stinner
ods created @staticmethod callable. Victor On Wed, Mar 31, 2021 at 4:19 PM Ethan Furman wrote: > > On 3/31/21 6:49 AM, Victor Stinner wrote: > > > tl; dr *Maybe* staticmethod could be modified to become callable? > > There have been other requests to make staticmeth

[Python-Dev] Re: [python-committers] NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-06 Thread Victor Stinner
Hi, About this very specific ABI issue, one long term solution would be to exclude the PyThreadState structure from the C API, to not rely on it the ABI level. I started to add getter functions in Python 3.9: PyThreadState_GetInterpreter(), PyThreadState_GetFrame() and PyThreadState_GetID(). I'm

[Python-Dev] Re: When we remove 'U' mode of open()?

2021-04-07 Thread Victor Stinner
Hi Inada-san, I'm +0 on removing again the flag, but I would prefer to not endorse the responsibility. I am already responsible for enough incompatible changes in Python 3.10 :-D Some context on this "U" open mode. The flag is accepted by many functions opening files. It is deprecated (emit Depre

[Python-Dev] Re: When we remove 'U' mode of open()?

2021-04-08 Thread Victor Stinner
U' in other places in Python 3.11, after 3.10 branch is > created (and master branch is renamed to main). > > On Thu, Apr 8, 2021 at 5:45 AM Brett Cannon wrote: > > > > > > > > On Wed, Apr 7, 2021 at 10:01 AM Serhiy Storchaka > > wrote: > >>

[Python-Dev] Please review the new Python Build System documentation

2021-04-12 Thread Victor Stinner
Hi, Please review this documentation: https://docs.python.org/dev/using/configure.html If you have comment, you can directly write a PR (please notify me by mentioning @vstinner in a comment). Or if you prefer, you can reply directly by email and I will try to address your remarks. -- Over

[Python-Dev] Re: Boundaries between numbers and identifiers

2021-04-13 Thread Victor Stinner
It would be useful to first estimate how many projects would be broken by such incompatible change (stricter syntax). Inada-san wrote https://github.com/methane/notes/blob/master/2020/wchar-cache/download_sdist.py to download source files using https://hugovk.github.io/top-pypi-packages/ API (top

[Python-Dev] Re: Boundaries between numbers and identifiers

2021-04-14 Thread Victor Stinner
Also, would it be possible to enhance to tokenizer to report a SyntaxWarning, rather than a SyntaxError? Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailm

[Python-Dev] Re: Revive PEP 396 -- Module Version Numbers ?

2021-04-14 Thread Victor Stinner
On Wed, Apr 14, 2021 at 7:48 AM Christopher Barker wrote: > So what do you'all think? After thirteen years, it would be nice to put this > to bed. There are two main use cases for versions: * Display them to the user * Compare versions to check if one is newer, older or the same I dislike usin

[Python-Dev] Re: gzip.py: allow deterministic compression (without time stamp)

2021-04-14 Thread Victor Stinner
Hi, gzip.NO_TIMESTAMP sounds like a good idea. But I'm not sure about changing the default behavior. I would prefer to leave it unchanged. I guess that your problem is that you don't access gzip directly, but uses a higher level API which doesn't give access to the timestamp parameter, like the t

[Python-Dev] Re: gzip.py: allow deterministic compression (without time stamp)

2021-04-15 Thread Victor Stinner
If gzip is modified to use SOURCE_DATE_EPOCH timestamp, you get a reproducible binary and you don't need to add a new constant ;-) SOURCE_DATE_EPOCH is a timestamp: number of seconds since Unix Epoch (January 1, 1970 at 00:00). Victor On Wed, Apr 14, 2021 at 8:15 PM wrote: > > The gzip specific

[Python-Dev] Re: Revive PEP 396 -- Module Version Numbers ?

2021-04-15 Thread Victor Stinner
Paul Bryan: > Seems like this is something that should make its way into stdlib? In the last 10 years, the trend is more to remove anything related to packaging *outside* the stdlib :-) Since it's evolving way faster than Python releases and the stdlib cannot be updated once installed, I don't thi

[Python-Dev] Re: gzip.py: allow deterministic compression (without time stamp)

2021-04-15 Thread Victor Stinner
, the Python stdlib already uses it in compileall and py_compile modules. Victor On Thu, Apr 15, 2021 at 12:34 PM Antoine Pitrou wrote: > > On Thu, 15 Apr 2021 11:28:03 +0200 > Victor Stinner wrote: > > If gzip is modified to use SOURCE_DATE_EPOCH timestamp, you get a > > re

[Python-Dev] Can we change python -W option and PYTHONWARNINGS to use a regex for the message?

2021-04-16 Thread Victor Stinner
Hi, I propose to change the -W command line option and the PYTHONWARNINGS environment variable to use the message as a regular expression in Python 3.10. Or does anyone have a reason to keep the current behavior as it is? I created https://bugs.python.org/issue43862 for this change. -- Python p

[Python-Dev] Re: Can we change python -W option and PYTHONWARNINGS to use a regex for the message?

2021-04-16 Thread Victor Stinner
/issue34624 Victor On Fri, Apr 16, 2021 at 3:22 PM Ivan Pozdeev via Python-Dev wrote: > > It'll probably be easier to change the warnings filter semantic > (https://docs.python.org/3/library/warnings.html#the-warnings-filter) > to which those values are directly passed. > > On 1

[Python-Dev] Re: How about using modern C++ in development of CPython ?

2021-04-16 Thread Victor Stinner
Hi, I used C++ in the past to write a small game. My experience was that the compilation was quite slow and many compiler errors were hard to understand. I love the fact the CPython is written in C because its build time is under one minute for a fresh build, and way faster for an incremental bui

[Python-Dev] Re: Question regarding the value of PyThreadState.thread_id

2021-04-17 Thread Victor Stinner
Hi, There are two reasons: * PyThread_get_thread_native_id() was only added recently (Python 3.8) * PyThread_get_thread_native_id() is not portable and not available on all platforms: the function is only declared if the PY_HAVE_THREAD_NATIVE_ID macro is defined. Maybe PyThreadState.thread_id co

[Python-Dev] Re: Should Python typing leverage PEP 263 as a pre-processor step?

2021-04-20 Thread Victor Stinner
I proposed PEP 511 "API for code transformers" for Python 3.6 (in 2016) and it was rejected: https://www.python.org/dev/peps/pep-0511/#rejection-notice """ This PEP was rejected by its author. This PEP was seen as blessing new Python-like programming languages which are close but incompatible wit

[Python-Dev] Re: Existing asyncio features scheduled for removal in Python 3.9 and 3.10

2021-04-27 Thread Victor Stinner
Hi, On Mon, Apr 26, 2021 at 10:51 PM Illia Volochii wrote: > There are a couple of uncompleted asyncio feature removals scheduled > for 3.9 and 3.10 releases. > It will be great if we either complete them or reschedule before the > 3.10 feature freeze. There are two stale pull requests related to

[Python-Dev] Re: In what tense should the changelog be written?

2021-04-30 Thread Victor Stinner
On Fri, Apr 30, 2021 at 6:22 AM Guido van Rossum wrote: > There’s something in the dev guide, but not about tense. Worth adding. (My > pet peeve is not to write “The foo module resets the bar state when spam > happens,” because that isn’t clear about whether that’s the before or after > behavio

[Python-Dev] Re: In what tense should the changelog be written?

2021-04-30 Thread Victor Stinner
On Fri, Apr 30, 2021 at 6:57 AM Larry Hastings wrote: > Your NEWS entry should be written in the present tense, and should start with > a verb: > > Add foo [...] > Change bar [...] > Remove bat [...] > Fix buffalo.spam [...] You might suggest using "now" when describing a behavior change: "foo n

[Python-Dev] Re: In what tense should the changelog be written?

2021-04-30 Thread Victor Stinner
By the way, https://docs.python.org/dev/whatsnew/3.10.html#improved-modules is inconsistent: "asyncio: Added missing ..." versus "base64: Add base64.b32hexencode() ..." You can also find such inconsistencies in "versionchanged" markups. Victor On Fri, Apr 30, 2021 at 5:01 AM Larry Hastings w

[Python-Dev] Re: [python-committers] [Release manager team communication] master blocked until 3.10 beta 1 is released

2021-05-03 Thread Victor Stinner
Hi, Please don't attempt to merge PRs until the GitHub issue described below is solved ;-) Pablo renamed the default "master" branch to "main" (in a live Twitch stream ;-)) but got a GitHub internal error! Maybe it's because the dialog announced that 1.4k+ pull requests and 700+ repositositories

[Python-Dev] Re: The repr of a sentinel

2021-05-14 Thread Victor Stinner
Hi Tal, Would it make sense to have an unique singleton for such sentinel, a built-in singleton like None or Ellipsis? I propose the name "Sentinel". Sentinel would be similar to None, but the main property would be that "Sentinel is None" is false :-) The stdlib contains tons of sentinels: * _

[Python-Dev] Re: The repr of a sentinel

2021-05-20 Thread Victor Stinner
IMO you should consider writing a PEP to enhance sentinels in Python, and maybe even provide a public API for sentinels in general. Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org ht

[Python-Dev] Re: GDB not breaking at the right place

2021-05-22 Thread Victor Stinner
I strongly suggest to only build Python with -O0 when using gdb. -Og enables too many optimizations which makes gdb less usable. Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https

[Python-Dev] Re: GDB not breaking at the right place

2021-05-24 Thread Victor Stinner
"Debugging" means many things. Python is built with -Og because it makes Python faster than -O0, and most developers debug Python code, not C code (in gdb). If you don't need to go up to the gdb/lldb level, -Og is fine. It would even make sense to build Python with -O3 in debug mode if you don't

[Python-Dev] Re: GDB not breaking at the right place

2021-05-24 Thread Victor Stinner
o obviously you might have to > debug C code. (I know that was the case when we introduced it, decades ago.) > If that's not the goal, then what is --py-debug used for? > > On Mon, May 24, 2021 at 3:35 AM Victor Stinner wrote: >> >> "Debugging" means ma

[Python-Dev] Re: Implementing async enumerate

2021-05-25 Thread Victor Stinner
Hi, I suggest to put such function in a Python module and publish it on PyPI. I'm sure that the module will quickly grow with more async flavor of existing functions. Victor On Tue, May 25, 2021 at 4:22 AM Filipe Alves Caixeta wrote: > > Hello, > > Python 3.10 comes with the built-in anext() bu

[Python-Dev] IRC #python-dev channel is now on Libera Chat (bye bye Freenode)

2021-05-26 Thread Victor Stinner
Hi, The #python-dev IRC channel moves from Freenode to Libera Chat: https://libera.chat/ I'm connected as vstinner, see you there ;-) Join the channel if you would like to talk about the development of Python itself (the VM, bytecode, stdlib, etc.)! Moreover, bots notifications (GitHub, buildbots

[Python-Dev] Re: IRC #python-dev channel is now on Libera Chat (bye bye Freenode)

2021-05-26 Thread Victor Stinner
2021 at 12:46 PM Victor Stinner wrote: > > Hi, > > The #python-dev IRC channel moves from Freenode to Libera Chat: > https://libera.chat/ I'm connected as vstinner, see you there ;-) Join > the channel if you would like to talk about the development of Python > itself

[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

2021-05-27 Thread Victor Stinner
Hi, The gilectomy was mentioned earlier in the thread. This project was created by a core dev who had the permission to push directly his work into the development branch, but it was done on a work. Another optimization project example was my experimental "FAT Python" project. The main idea was t

[Python-Dev] Need help to debug a ssl crash on Windows which prevents merging PRs

2021-05-28 Thread Victor Stinner
Hi, In the 3.10 branch, it became really hard to merge PRs because the following ssl crashs on Windows: https://bugs.python.org/issue44252 It has a failure rate 1/2 (on average) on the "Windows x86" and "Windows x64" jobs of GitHub Action and on the Win32 and Win64 jobs of the Azure Pipelines. I

<    8   9   10   11   12   13   14   15   16   17   >