[Python-Dev] Star the CPython GitHub project if you like Python!

2017-06-30 Thread Victor Stinner
Hi, GitHub has a showcase page of hosted programming languages: https://github.com/showcases/programming-languages Python is only #11 with 8,539 stars, behind PHP and Ruby! Hey, you should "like" ("star"?) the CPython project if you like Python! https://github.com/python/cpython/ Clic

Re: [Python-Dev] Summary of Python tracker Issues

2017-07-01 Thread Victor Stinner
Le 1 juil. 2017 10:18 AM, "Serhiy Storchaka" a écrit : Victor closed a half of hundred his issues. Let me elaborate :-) I am still learning the new GitHub workflow, and it's common that I forget to close issues after merging a change. I had many issues that I forgot to update since we moved to

Re: [Python-Dev] Lazy_loading to pickle/unpickle_pure?

2017-07-01 Thread Victor Stinner
What is lazy loading? How does it work? Victor Le 1 juil. 2017 6:02 PM, "Bhavishya" a écrit : > Hi, > I added lazy_loading in pickle.py.Here's some statistics if you consider > them of any importance: > > 1)Unpickle_pure: > Lazy -> unpickle_pure_python: Mean +- std dev: 728 us +- 24 us > > Orig

Re: [Python-Dev] 64 bit units in PyLong

2017-07-03 Thread Victor Stinner
2017-07-03 6:52 GMT+02:00 Siyuan Ren : > The current PyLong implementation represents arbitrary precision integers in > units of 15 or 30 bits. I presume the purpose is to avoid overflow in > addition , subtraction and multiplication. But compilers these days offer > intrinsics that allow one to ac

Re: [Python-Dev] Remove own implementation for thread-local storage

2017-07-03 Thread Victor Stinner
I'm in favor of removing it. I know that it confused people many times, they look at this fallback and found an issue, whereas I'm not aware of any platform using this fallback anymore. Can you please write a PR just to remove this fallback? We can merge it and then check buildbots :-) So in the w

Re: [Python-Dev] Remove own implementation for thread-local storage

2017-07-03 Thread Victor Stinner
> > Regards > > Antoine. > > > On Mon, 3 Jul 2017 10:07:06 +0200 > Victor Stinner wrote: > >> I'm in favor of removing it. I know that it confused people many >> times, they look at this fallback and found an issue, whereas I'm not >> aware of any pl

[Python-Dev] Need help to review a test_nntplib enhancement:

2017-07-03 Thread Victor Stinner
Hi, Sometimes, for an unknown reason, test_nntplib fails randomly: http://bugs.python.org/issue19613 Martin Panter wrote a patch, but since I don't know how to reproduce the bug, I'm unable to test it. Moreover, I don't know nntplib nor test_nntplib, so I don't feel able to review it. Sadly,

Re: [Python-Dev] Remove own implementation for thread-local storage

2017-07-03 Thread Victor Stinner
>> I'd propose removing code which I think out-of-date. Already done! https://github.com/python/cpython/commit/aa0aa0492c5fffe750a26d2ab13737a1a6d7d63c (and no buildbot complained). Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.p

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-07-04 Thread Victor Stinner
test.bisectcmd ... instead of ./python -m tes.bisect ... See http://bugs.python.org/issue30843 for more information. Victor 2017-06-16 18:05 GMT+02:00 Victor Stinner : > Hi, > > Last weeks, I worked on a new tool to bisect failing tests because > it's painful to bisect manually reference l

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-07-04 Thread Victor Stinner
2017-07-04 12:52 GMT+02:00 Nick Coghlan : > I know it's longer, but perhaps it would make sense to put the > bisection helper under "python -m test.support.bisect" in both Python > 2 & 3? For me, test.support is a toolkit to *write* tests, not really to run tests. I don't really care where my bis

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-07-04 Thread Victor Stinner
2017-07-04 13:22 GMT+02:00 Nick Coghlan : > That means if test.bisect is shadowing the top level bisect module > when backported, it suggests that the test.regrtest directory is > ending up on sys.path for the affected test run (e.g. because the > tests were run as "python Lib/test/regrtest.py" rat

Re: [Python-Dev] Star the CPython GitHub project if you like Python!

2017-07-04 Thread Victor Stinner
4 days later, we got +2,389 new stars, thank you! (8,539 => 10,928) Python moved from the 11th place to the 9th, before Elixir and Julia. Python is still behind Ruby (12,511) and PHP (12,318), but it's already much better than before! Victor 2017-06-30 15:59 GMT+02:00 Victor Stinne

Re: [Python-Dev] Buildbot report (almost July)

2017-07-04 Thread Victor Stinner
2017-06-29 17:09 GMT+02:00 Victor Stinner : > Correct me if I'm wrong, but, for the first time, *all reference > leaks* have been fixed on *all branches* (2.7, 3.5, 3.6 and master), > on *Linux and Windows*! Before, we mostly focused on the master branch > (called "default&q

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-07-04 Thread Victor Stinner
2017-07-04 16:27 GMT+02:00 Nick Coghlan : > That concern just doesn't apply to the *stdlib* modules doing a normal > top-level "import bisect". Hum, ok. I created a PR which removes '' and Lib/test/ from sys.path, and rename again test.bisectcmd to test.bisect. Would you mind to review it? https:

Re: [Python-Dev] Pure pickle bechmark.

2017-07-09 Thread Victor Stinner
Wait. Are we talking about the C accelerator or the pure Python implementation of pickle on Python 3? Victor Le 10 juil. 2017 01:19, "INADA Naoki" a écrit : > I don't know this is relating to your case. > > When I saw Victor's report [1], I researched why Python 3 is slower than > Python 2 on u

Re: [Python-Dev] Pure pickle bechmark.

2017-07-09 Thread Victor Stinner
Please explain how to reproduce your benchmark. Maybe write a shell script? Victor Le 9 juil. 2017 17:49, "Bhavishya" a écrit : > Hello, > > 1).I was going through the code of *python pickle* to search any > optimization possibility.But the only thing that I found very alarming was > again the

Re: [Python-Dev] Pure pickle bechmark.

2017-07-09 Thread Victor Stinner
t could be a case with pickle.py > too. And thus tried adding the above patch to Lib/pickle.py to measure the > initial import time. > > I haven't tried it for any practical use-case. > > > On Mon, Jul 10, 2017 at 5:27 AM, Victor Stinner > wrote: >> >> Sorry,

Re: [Python-Dev] [python-committers] Should I make a 3.4.7rc1 next weekend?

2017-07-12 Thread Victor Stinner
I would love to have a new 3.4 release including all security fixes, sure! It would reduce the number of known vulnerability in Python 3.4: http://python-security.readthedocs.io/vulnerabilities.html 2017-07-12 15:09 GMT+02:00 Larry Hastings : > After a flurry of accepted PRs, I have now accrue

[Python-Dev] Articles on my contributions to CPython during 2017 Q1 and Q2

2017-07-13 Thread Victor Stinner
Hi, I wrote a serie of new articles on my contributions to CPython during 2017 Q1 and Q2. "My contributions to CPython during 2017 Q1" https://haypo.github.io/contrib-cpython-2017q1.html "New Python test.bisect tool" https://haypo.github.io/python-test-bisect.html "Work on Python buildbots, 201

Re: [Python-Dev] Articles on my contributions to CPython during 2017 Q1 and Q2

2017-07-14 Thread Victor Stinner
2017-07-14 13:37 GMT+02:00 Ben Hoyt : > Wow, amazing work. The Stinnerbot strikes again! Thanks. > A lot of great optimizations and bugfixes. Speaking of optimizations, I just > wrote some code which takes 12s on Python 2.7 and 5s on Python 3.5. so we're > doing something right! I might post abou

Re: [Python-Dev] Articles on my contributions to CPython during 2017 Q1 and Q2

2017-07-16 Thread Victor Stinner
e "-m test" with "-m test.bisect" in your command line, and you are done. Victor 2017-07-16 17:30 GMT+02:00 francismb : > Hi Victor, > > On 07/13/2017 05:33 PM, Victor Stinner wrote: >> Hi, >> >> I wrote a serie of new articles on my contributions to C

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Victor Stinner
2017-07-17 16:56 GMT+02:00 Facundo Batista : > My experience inside Canonical is that golang stole a lot of "codebase > share" from Python, and (others and mine) talks hit two walls, mainly: > one is memory consumption, and the other is startup time. > > So yes, startup time is important for user-f

Re: [Python-Dev] Impact of Namedtuple on startup time

2017-07-17 Thread Victor Stinner
2017-07-17 18:13 GMT+02:00 Gregory P. Smith : > I get that namedtuple ._source is a public API. We may need to keep it. If > so, that just means revisiting lazily generating it as a property - > issue19640. I agree. Technically speaking, optimizing namedtuple doesn't have to mean "remove the _sour

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Victor Stinner
[Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability 2017-07-18 18:08 GMT+02:00 Ethan Furman : > Nick Coughlan: > - >> >> As another example of this: while trading the global import lock for >> per-module locks eliminated most of the old import deadlocks, (...) Mi

Re: [Python-Dev] Design Philosophy: Performance vs Robustness/Maintainability

2017-07-18 Thread Victor Stinner
2017-07-18 18:08 GMT+02:00 Ethan Furman : > Raymond Hettinger: > - >> And complexity leads to bugs >> (the C >> optimization of random number seeding caused a major bug in the 3.6.0 >> release Hum, I guess that Raymond is referring to http://bugs.python.org/issue29085 This regress

[Python-Dev] Python startup time

2017-07-19 Thread Victor Stinner
perf command -- ...". == CPython core developers don't care? no, they do care == Christian Heimes, Naoki INADA, Serhiy Storchaka, Yury Selivanov, me (Victor Stinner) and other core developers made multiple changes last years to reduce the number of imports at startup, optimize impotlib,

Re: [Python-Dev] Python startup time

2017-07-19 Thread Victor Stinner
2017-07-19 15:22 GMT+02:00 Oleg Broytman : > On Wed, Jul 19, 2017 at 02:59:52PM +0200, Victor Stinner > wrote: >> "Python is very slow to start on Windows 7" >> https://stackoverflow.com/questions/29997274/python-is-very-slow-to-start-on-windows-7 > >Howe

Re: [Python-Dev] Python startup time

2017-07-20 Thread Victor Stinner
Hi, I applied the patch above to count the number of times that Python is run. Running the Python test suite with "./python -m test -j0 -rW" runs Python 2,256 times. Honestly, I expected more. I'm running tests with Python compiled in debug mode. And in debug mode, Python startup time is much wor

Re: [Python-Dev] Python startup time

2017-07-20 Thread Victor Stinner
2017-07-20 19:09 GMT+02:00 Cesare Di Mauro : > I assume that Python loads compiled (.pyc and/or .pyo) from the stdlib. > That's something that also influences the startup time (compiling source vs > loading pre-compiled modules). My benchmark was "python3 -m perf command -- python3 -c pass": I d

[Python-Dev] Need help to fix urllib(.parse) vulnerabilities

2017-07-21 Thread Victor Stinner
Hi, Recently, two security vulnerabilities were reported in the urllib module: https://bugs.python.org/issue30500 http://python-security.readthedocs.io/vuln/bpo-30500_urllib_connects_to_a_wrong_host.html#bpo-30500-urllib-connects-to-a-wrong-host => already fixed in Python 3.6.2 https://bugs.pyth

Re: [Python-Dev] Need help to fix urllib(.parse) vulnerabilities

2017-07-21 Thread Victor Stinner
2017-07-21 12:02 GMT+02:00 Victor Stinner : > https://bugs.python.org/issue29606 > http://python-security.readthedocs.io/vuln/urllib_ftp_protocol_stream_injection.html#urllib-ftp-protocol-stream-injection > => not fixed yet Ok, I more concrete problem. To fix the "urllib FTP"

Re: [Python-Dev] Need help to fix urllib(.parse) vulnerabilities

2017-07-22 Thread Victor Stinner
Le 22 juil. 2017 8:04 AM, "Serhiy Storchaka" a écrit : I think the only reliable way of fixing the vulnerability is rejecting or escaping (as specified in RFC 2640) CR and LF inside sent lines. Adding the support of RFC 2640 is a new feature and can be added only in 3.7. And this feature should b

Re: [Python-Dev] Need help to fix urllib(.parse) vulnerabilities

2017-07-22 Thread Victor Stinner
I consider that it is a security vulneraibility and so should be fixed in all supported branches including 3.3 and 3.4. If someone is blocked for a legit usecase, an old Python version can be used until we decide how to handle it. I concur with you, I don't think that anyone uses filenames contai

Re: [Python-Dev] startup time repeated? why not daemon

2017-07-23 Thread Victor Stinner
We already did that. See _bootlocale for example. (Maybe also _collecctions_abc?) Victor Le 22 juil. 2017 07:20, "Chris Jerdonek" a écrit : > On Fri, Jul 21, 2017 at 9:52 AM, Brett Cannon wrote: > > On Thu, 20 Jul 2017 at 22:11 Chris Jerdonek > > wrote: > >> On Thu, Jul 20, 2017 at 8:49 PM, N

[Python-Dev] Is Windows XP still supported on Python 2.7?

2017-07-24 Thread Victor Stinner
Hi, We have a Windows XP buildbot for Python 2.7, run by David Bolen: http://buildbot.python.org/all/builders/x86%20Windows%20XP%202.7/ test_bsddb3 fails randomly on this buildbot: http://bugs.python.org/issue30778 But Windows XP clearly reached its end-of-life, Microsoft doesn't support it anym

Re: [Python-Dev] Is Windows XP still supported on Python 2.7?

2017-07-24 Thread Victor Stinner
2017-07-24 11:38 GMT+02:00 Alex Walters : > The promise that PEP-11 is making is that as long as a python was released > while Microsoft still supported that OS, and that python is still supported, > there will still be a python that works for you. So, yes, Windows XP is > long since unsupported b

Re: [Python-Dev] Is Windows XP still supported on Python 2.7?

2017-07-24 Thread Victor Stinner
2017-07-24 19:05 GMT+02:00 Zachary Ware : > In this case, considering that it's a test of a > 2.x-only module on an out-of-vendor-support OS, skipping the tests > (possibly even the entirety of test_bsddb3) on XP sounds just fine to > me. Oh ok. Since Terry and you agree on that, I will skip the t

Re: [Python-Dev] [python-committers] RELEASED] Python 3.4.7rc1 and Python 3.5.4rc1 are now available

2017-07-25 Thread Victor Stinner
2017-07-25 10:37 GMT+02:00 Larry Hastings : > On behalf of the Python development community and the Python 3.4 and Python > 3.5 release teams, I'm relieved to announce the availability of Python > 3.4.7rc1 and Python 3.5.4rc1. I checked for known security vulnerabilities: except of the issue #2960

Re: [Python-Dev] first post introduction and question regarding lto

2017-08-07 Thread Victor Stinner
I don't think that PGO compilation itself is slow. Basically, I expect that it only doubles the compilation time, but compiling Python takes less than 1 minute usually. The slow part is the profiling task: run the full Python test suite, which takes at least 20 minutes. The tests must be run sequen

Re: [Python-Dev] [ANN] Daily Windows builds of Python 3.x

2017-08-08 Thread Victor Stinner
Thank you! I recall that we discussed that, but I understood that you was too busy to implement the idea. No, you didn't forget and you made it! ;-) Victor 2017-08-08 18:21 GMT+02:00 Steve Dower : > Hi all > > As part of a deal with Zach Ware at PyCon, I agreed that if he removed the > Subversio

Re: [Python-Dev] [RELEASED] Python 3.5.4 is now available

2017-08-08 Thread Victor Stinner
I updated some websites and services for the 3.5.4 release: * Status of Python branches in the devguide: https://devguide.python.org/#status-of-python-branches * Python security vulnerabilities: http://python-security.readthedocs.io/vulnerabilities.html * I removed all Python 3.5 buildbots:

Re: [Python-Dev] first post introduction and question regarding lto

2017-08-09 Thread Victor Stinner
There is already a ./configure --with-lto flag, why not using it? I'm using --with-lto without PGO for months, I never noticed that the option is fully ignored! Victor 2017-08-09 9:52 GMT+02:00 Antoine Pitrou : > On Wed, 9 Aug 2017 13:36:28 +1000 > Nick Coghlan wrote: >> On 8 August 2017 at 10:

Re: [Python-Dev] first post introduction and question regarding lto

2017-08-09 Thread Victor Stinner
2017-08-09 11:22 GMT+02:00 Antoine Pitrou : > What are the reasons it is ignored? IIRC some compilers have buggy LTO > support and it can lead to crashes during compilation. Issues with LTO: http://bugs.python.org/issue28032 http://bugs.python.org/issue28605 But since --with-lto is now an opt-in

[Python-Dev] socketserver ForkingMixin waiting for child processes

2017-08-11 Thread Victor Stinner
Hi, I'm working on reducing the failure rate of Python CIs (Travis CI, AppVeyor, buildbots). For that, I'm trying to reduce test side effects using "environment altered" warnings. This week, I worked on support.reap_children() which detects leaked child processes (usually created with os.fork()).

Re: [Python-Dev] Git and Mercurial Security Update

2017-08-14 Thread Victor Stinner
By the way, I suggest you to enable 2-factor authentication in GitHub. You can use FreeOTP on your smartphone, or buy a yubikey nano, for example. Victor Le 14 août 2017 03:11, "Cheryl Sabella" a écrit : > http://www.esecurityplanet.com/threats/git-svn-and- > mercurial-open-source-version-contr

Re: [Python-Dev] socketserver ForkingMixin waiting for child processes

2017-08-16 Thread Victor Stinner
Hi, The first bug was that test_socketserver "leaked" child processes: it means that socketserver API creates zombie processes depending how long the child processes take to complete. If you want to backport my change waiting until child processes complete, you need to fix the bug differently, so

[Python-Dev] Buildbot report, August 2017

2017-08-18 Thread Victor Stinner
Hi, Here is a quick report of what changed recently on buildbots. == pythoninfo == I added a new "python3 -m test.pythoninfo" command which is now run on Travis CI, AppVeyor and buildbots. https://bugs.python.org/issue30871 This command dumps various informations to help debugging test fa

Re: [Python-Dev] socketserver ForkingMixin waiting for child processes

2017-08-18 Thread Victor Stinner
2017-08-12 0:34 GMT+02:00 Ryan Smith-Roberts : > Since ThreadingMixIn also leaks threads, > server_close() could grow a timeout flag (following the socket module > timeout convention) and maybe a terminate boolean. ThreadingMixIn could then > also be fixed. I'm not sure how useful that is though, s

Re: [Python-Dev] [python-committers] Python 3.3.7 release schedule and end-of-life

2017-08-31 Thread Victor Stinner
Hello, 2017-07-15 23:51 GMT+02:00 Ned Deily : > To that end, I would like to schedule its next, and hopefully final, > security-fix release to coincide with the already announced 3.4.7 > security-fix release. In particular, we'll plan to tag and release 3.3.7rc1 > on Monday 2017-07-24 (UTC) and

[Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Victor Stinner
Hi, When I go to http://bugs.python.org/ Firefox warns me that the form on the left to login (user, password) sends data in clear text (HTTP). Ok, I switch manually to HTTPS: add "s" in "http://"; of the URL. I log in. I go to an issue using HTTPS like https://bugs.python.org/issue31250 I modi

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Victor Stinner
2017-09-01 15:36 GMT+02:00 Antoine Pitrou : > And by the way the problem goes away if you use the "HTTPS Everywhere" > plugin for Firefox. I do have "HTTPS Everywhere" Firefox plugin version 2017.8.31 (so it seems very recent), but it displayed as "obsolete" ("obsolète" in french). I'm using Firef

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Victor Stinner
2017-09-01 16:34 GMT+02:00 Antoine Pitrou : > I'm using Firefox 55 on Ubuntu 16.04 and it works here. You may be > misunderstading what happens :-) Maybe I misunderstood you when you wrote: > And by the way the problem goes away if you use the "HTTPS Everywhere" > plugin for Firefox. Try for ex

Re: [Python-Dev] HTTPS on bugs.python.org

2017-09-01 Thread Victor Stinner
2017-09-01 19:06 GMT+02:00 Antoine Pitrou : > That's surprising. It's definitely part of the standard rules (enabled > by default): > https://www.eff.org/https-everywhere/atlas/domains/python.org.html Maybe the plugin is also broken, as my setup. Maybe it's related to the recent "multiprocess" ma

Re: [Python-Dev] Compiling without multithreading support -- still useful?

2017-09-05 Thread Victor Stinner
I proposed to drop the --without-threads option multiple times. I worked on tiny and cheap embedded devices and we used Python *with* threads for concurrency. Many Python features require threads, like asyncio and multiprocessing. Also subprocess.communicate() on Windows, no? I'm strongly in favor

Re: [Python-Dev] Compiling without multithreading support -- still useful?

2017-09-06 Thread Victor Stinner
2017-09-06 22:19 GMT+02:00 Berker Peksağ : > Do we still have buildbots for testing the --without-threads option? We had such buildbot once, but it's gone. I just removed its unused class from the buildbot configuration: https://github.com/python/buildmaster-config/commit/091f52aa05a8977966796ba3e

[Python-Dev] New C API not leaking implementation details: an usable stable ABI

2017-09-06 Thread Victor Stinner
Hi, I am currently at a CPython sprint 2017 at Facebook. We are discussing my idea of writing a new C API for CPython hiding implementation details and replacing macros with function calls. I wrote a short blog post to explain the issue of the current API, the link between the API and the ABI, an

[Python-Dev] Re: What is __int__ still useful for?

2021-10-13 Thread Victor Stinner
Hi Antoine, I have a lot of troubles to reminder how Python converts numbers, I collected notes about the Python "number tower" and the C implementation: https://pythondev.readthedocs.io/numbers.html Honestly, I don't understand well the difference between __int__() and __index__(). * https://do

[Python-Dev] PEP 670: Convert macros to functions in the Python C API

2021-10-19 Thread Victor Stinner
Python C API Author: Erlend Egeberg Aasland , Victor Stinner Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 19-Oct-2021 Python-Version: 3.11 Abstract Convert macros to static inline functions or regular functions. Remove the return value of macros having a

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-10-19 Thread Victor Stinner
Extra info that I didn't put in the PEP to keep the PEP short. Since Python 3.8, multiple macros have already been converted, including Py_INCREF() and Py_TYPE() which are very commonly used and so matter for Python performance. Macros converted to static inline functions: * Py_INCREF(), Py_DECR

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-10-19 Thread Victor Stinner
One of my motivation to write this PEP was decide how to solve the issue: "[C API] Disallow using PyFloat_AS_DOUBLE() as l-value" https://bugs.python.org/issue45476 I proposed two fixes: * Convert macros to static inline functions: https://github.com/python/cpython/pull/28961 * Fix the macro, add

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-10-21 Thread Victor Stinner
On Wed, Oct 20, 2021 at 10:58 AM Petr Viktorin wrote: > I think this info should be in the PEP. Ok, we added (and completed) the list to the PEP: https://www.python.org/dev/peps/pep-0670/#macros-converted-to-functions-since-python-3-8 > If the PEP is rejected, would all these previous changes ne

[Python-Dev] Re: PEP 467: Minor bytes and bytearray improvements

2021-11-08 Thread Victor Stinner
The ascii() constructor is not well specified by the PEP. There are only a few examples. I don't understand how it's supposed by be implemented. Would you mind to elaborate its specification? Is it implement "like" ascii(obj).encode("ascii") but with minor changes? What changes? Victor __

[Python-Dev] Re: Oh wow, this is really impressive

2021-11-08 Thread Victor Stinner
What's New in Python 3.10 lists other suggestions and enhanced error messages: https://docs.python.org/dev/whatsnew/3.10.html#better-error-messages Victor On Fri, Oct 29, 2021 at 7:22 PM Steven D'Aprano wrote: > > I was using Python 3.10 and got this NameError when I mistyped a name: > > NameErr

[Python-Dev] Re: PEP 467: Minor bytes and bytearray improvements

2021-11-09 Thread Victor Stinner
On Mon, Nov 8, 2021 at 8:21 PM Ethan Furman wrote: > The difference with the built-in ascii is the absence of extra quotes and the > `b` indicator when a string is used: > > ``` > >>> u_var = u'abc' > >>> bytes.ascii(u_var) > b'abc' What about bytes, bytearray and memoryview? What is the expec

[Python-Dev] Re: Proposal: Allow non-default after default arguments

2021-11-09 Thread Victor Stinner
IMO it was a bad idea to merge 2 ncurses C functions into a single Python function. In the C API, there are two different functions: * mvwadd_wch(win, y, x, char): 4 arguments * wadd_wch(win, char): 2 arguments The Python curses module could/can have a separated function when (y, x) arguments are

[Python-Dev] Re: Having Sorted Containers in stdlib?

2021-11-10 Thread Victor Stinner
IMO if someone is motivated to get a new container type in Python, a PEP is required. A PEP has been written for the new removeprefix() and removesuffix() methods which are way simpler ;-) I expect many questions on corner cases for a sorted container type. Having a reference implementation, eve

[Python-Dev] Remove asyncore, asynchat and smtpd modules

2021-11-11 Thread Victor Stinner
Hi, The asyncore module is a very old module of the Python stdlib for asynchronous programming, usually to handle network sockets concurrently. It's a common event loop, but its design has many flaws. The asyncio module was added to Python 3.4 with a well designed architecture. Twisted developers

[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules

2021-11-12 Thread Victor Stinner
> > It was decided to start deprecating the asyncore, asynchat and smtpd > > modules in Python 3.6 released in 2016, 5 years ago. Python 3.10 emits > > DeprecationWarning. > > Wait, only Python 3.10? > According to the policy, the warning should be there for *at least* two > releases. (That's a min

[Python-Dev] Re: Do we need to remove everything that's deprecated?

2021-11-12 Thread Victor Stinner
> The current backwards compatibility policy (PEP 387) sets a *minimum* > timeline for deprecations and removals -- "deprecation period must last > at least two years." About the PEP 387 process and the 3 examples. On Fri, Nov 12, 2021 at 11:58 AM Petr Viktorin wrote: >AttributeError: module

[Python-Dev] Re: Do we need to remove everything that's deprecated?

2021-11-12 Thread Victor Stinner
For me, deprecated functions cause me a lot of thinking when I met them as a Python maintainer and as a Python user. Why is it still there? What is its purpose? Is there a better alternative? It's related to the Chesterton's fence principle. Sometimes, reading the doc is enough. Sometimes, I have t

[Python-Dev] Re: Update colorsys with YUV conversions?

2021-11-12 Thread Victor Stinner
I guess that you should agree on constants and then stick to them. Otherwise, we might have to add a parameter later to chose the conversion standard. The PR uses "ATSC BT.709 standard constant". Wikipedia says: "When encoding Y’CBCR video, BT.709 creates gamma-encoded luma (Y’) using matrix coef

[Python-Dev] Re: Do we need to remove everything that's deprecated?

2021-11-14 Thread Victor Stinner
On Sun, Nov 14, 2021 at 6:34 PM Eric V. Smith wrote: > On second thought, I guess the existing policy already does this. Maybe > we should make it more than 2 versions for deprecations? I've written > libraries where I support 4 or 5 released versions. Although maybe I > should just trim that back

[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules

2021-11-15 Thread Victor Stinner
On Tue, Nov 16, 2021 at 1:15 AM Brett Cannon wrote: > But have they been raising exceptions for two releases? As I wrote previously, the DeprecationWarning warning is only emitted at runtime since Python 3.10. Since my PR got 5 approvals, I just merged it: https://github.com/python/cpython/pull/

[Python-Dev] Re: Remove asyncore, asynchat and smtpd modules

2021-11-16 Thread Victor Stinner
I created https://github.com/python/steering-council/issues/86 to ask for a SC exception. Victor On Tue, Nov 16, 2021 at 8:11 PM Brett Cannon wrote: > > > > On Tue, Nov 16, 2021 at 3:05 AM Petr Viktorin wrote: >> >> On 12. 11. 21 13:09, Victor Stinner wrote: >&g

[Python-Dev] Re: Do we need to remove everything that's deprecated?

2021-11-18 Thread Victor Stinner
Maybe once a function is deprecated in Python, pyupgrade should be updated? I mean, more collaboration between Python core devs and the pyupgrade development. https://github.com/asottile/pyupgrade Victor On Thu, Nov 18, 2021 at 8:39 AM Jeremiah Paige wrote: > > I’ve seen a few people in this th

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-11-23 Thread Victor Stinner
Hi Antoine, I completed the PEP: https://python.github.io/peps/pep-0670/ * Add benchmarks on a Python debug build: (1) macros vs static inline functions and (2) gcc -O0 vs gcc -Og * Elaborate the Debug Build section * Explain why the "keep macros" idea was rejected Diff: https://github.com/pyth

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-11-24 Thread Victor Stinner
On Wed, Nov 24, 2021 at 10:59 AM Petr Viktorin wrote: > Are there more macros that are yet to be converted to macros, I suppose that you mean "to be converted to functions". Yes, there are many, it's the purpose of the PEP. I didn't provide a list. I would prefer to do it on a case by case basis

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-11-24 Thread Victor Stinner
On Wed, Nov 24, 2021 at 10:59 AM Petr Viktorin wrote: > Since this is about converting existing macros (and not writing new > ones), can you talk about which of the "macro pitfalls" apply to the > macros in CPython that were/will be changed? The PEP 670 lists many pitfalls affecting existing macr

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-11-24 Thread Victor Stinner
On Wed, Nov 24, 2021 at 2:18 PM Petr Viktorin wrote: > >> The "Backwards Compatibility" section is very small. Can you give a list > >> of macros which lost/will lose "return values"? > > > > https://bugs.python.org/issue45476 lists many of them. See also: > > https://github.com/python/cpython/pul

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

2021-11-24 Thread Victor Stinner
24, 2021 at 12:27 AM Guido van Rossum wrote: > > On Tue, Nov 23, 2021 at 3:15 PM Antoine Pitrou wrote: >> >> On Tue, 23 Nov 2021 18:00:28 +0100 >> Victor Stinner wrote: >> >> > I didn't run benchmarks on Python built in release mode, since gcc -O3 >&g

[Python-Dev] Re: Optimizing literal comparisons and contains

2021-11-29 Thread Victor Stinner
You should consider "no longer have to justify why it's not optimized" as a clear benefit of making this change :-) This optimization is proposed once a year for many years... For me, any possible compilation-ahead optimization (which doesn't break the Python semantics) is worth it ;-) It's done o

[Python-Dev] Re: Optimizing literal comparisons and contains

2021-11-29 Thread Victor Stinner
optimized neither. Oh. I made the assumptions that other operations were already optimized. Victor On Mon, Nov 29, 2021 at 6:11 PM Serhiy Storchaka wrote: > > 29.11.21 18:36, Victor Stinner пише: > > You should consider "no longer have to justify why it's not optimize

[Python-Dev] Re: Optimizing literal comparisons and contains

2021-11-30 Thread Victor Stinner
If someone wants to experiment such optimization, there is no need to modify the Python internal optimizer, it can be done externally: https://faster-cpython.readthedocs.io/ast_optimizer.html For example, I implemented many optimizations like constant propagation and loop unrolling in my old AST f

[Python-Dev] PEP 674: Disallow using macros as l-value

2021-11-30 Thread Victor Stinner
for older Python versions. I already prepared major projects like Cython and numpy for these changes (in total, 14 impacted projects have already been updated). Victor --- PEP: 674 Title: Disallow using macros as l-value Author: Victor Stinner Status: Draft Type: Standards Track Content-Type:

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-11-30 Thread Victor Stinner
On Tue, Nov 30, 2021 at 7:34 PM Guido van Rossum wrote: > How about *not* asking for an exception and just following the PEP 387 > process? Is that really too burdensome? The Backward Compatibility section gives an explanation: "This change does not follow the PEP 387 deprecation process. There

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-12-01 Thread Victor Stinner
On Tue, Nov 30, 2021 at 7:52 PM Victor Stinner wrote: > For this specific PEP changes, I consider that the number of impacted > projects is low enough to skip a deprecation process: only 4 projects > are known to be impacted. One year ago (Python 3.10), 16 were > impacted, and 12

[Python-Dev] Re: Oh look, I've been subscribed to python/issues-test-2 notifications again

2021-12-02 Thread Victor Stinner
Hi Ezio, What is the status of migrating Python issues to GitHub? Is it done? If not, what are remaining issues? Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.

[Python-Dev] Tool to search in the source code of PyPI top 5000 projects

2021-12-02 Thread Victor Stinner
Hi, I wrote two scripts based on the work of INADA-san's work to (1) download the source code of the PyPI top 5000 projects (2) search for a regex in these projects (compressed source archives). You can use these tools if you work on an incompatible Python or C API change to estimate how many pro

[Python-Dev] Re: Tool to search in the source code of PyPI top 5000 projects

2021-12-03 Thread Victor Stinner
likely that old versions are also affected. Victor On Fri, Dec 3, 2021 at 8:35 AM Michał Górny wrote: > > On Fri, 2021-12-03 at 00:44 +0100, Victor Stinner wrote: > > I wrote two scripts based on the work of INADA-san's work to (1) > > download the source code of the PyPI top 5000

[Python-Dev] Re: Tool to search in the source code of PyPI top 5000 projects

2021-12-03 Thread Victor Stinner
Hi Steve, I completely agree with all you said ;-) I will not debate here if incompatible changes are worth it or not, this topic was discussed recently in another thread. On Fri, Dec 3, 2021 at 2:56 PM Steve Dower wrote: > FTR, I don't consider the top projects on PyPI to be representative of

[Python-Dev] peps, devguide, voters repositories: master branch renamed to main

2021-12-04 Thread Victor Stinner
Hi, The "master" branch of the following Python GitHub repositories have been renamed to "main": * devguide * peps * voters For the rationale of the rename, see: https://sfconservancy.org/news/2020/jun/23/gitbranchname/ If you already have a Git checkout of one of these repositories, you can re

[Python-Dev] Re: Clarification regarding Stable ABI and _Py_*

2021-12-06 Thread Victor Stinner
On Mon, Dec 6, 2021 at 9:54 PM Guido van Rossum wrote: > Also, it looks like Mark is proposing to *remove* _PyObject_GC_Malloc from > stable_abi.txt In Python 3.2, _PyObject_GC_Malloc() is implemented as a function. PyObject_GC_New() macro calls _PyObject_GC_New() function. Internally, PyType_Ge

[Python-Dev] Re: PEP 669: Low Impact Monitoring for CPython

2021-12-07 Thread Victor Stinner
Interesting! Some remarks about the proposed API. On Tue, Dec 7, 2021 at 4:58 PM Mark Shannon wrote: > There is no change to the language and it adds 7 functions to the sys module, > so shouldn't be too intrusive for those of who aren't planning on > implementing any of those tools. Where are

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-12-07 Thread Victor Stinner
On Tue, Dec 7, 2021 at 3:43 PM Petr Viktorin wrote: > If we would deprecate using Py_REFCNT as l-value in the docs, but wait > with the conversion until it was *actually* needed, we would not lose > anything: > (...) > ## CPython nogil fork > > In CPython, we cannot change structs that are part of

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-12-07 Thread Victor Stinner
ue, Dec 7, 2021 at 4:41 PM Antoine Pitrou wrote: > > On Tue, 7 Dec 2021 15:39:25 +0100 > Petr Viktorin wrote: > > > On 30. 11. 21 19:52, Victor Stinner wrote: > > > On Tue, Nov 30, 2021 at 7:34 PM Guido van Rossum wrote: > > >> How about *not*

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-12-07 Thread Victor Stinner
ge. > > > > On Tue, 7 Dec 2021 at 12:40, Antoine Pitrou wrote: >> >> On Tue, 7 Dec 2021 15:39:25 +0100 >> Petr Viktorin wrote: >> >> > On 30. 11. 21 19:52, Victor Stinner wrote: >> > > On Tue, Nov 30, 2021 at 7:34 PM Guido van Rossum >&

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-12-07 Thread Victor Stinner
For me, HPy is the only valid stable API and stable ABI in the long term which is efficient on any Python implementation. Its design is very different than the C API: HPy avoids all C API design mistakes, it doesn't leak any implementation detail. HPy can already be used today on CPython, even if

[Python-Dev] Re: PEP 674: Disallow using macros as l-value

2021-12-07 Thread Victor Stinner
Are you talking about gcc -Wignored-qualifiers? It seems like such warning is only emitted where the function is *defined*, not where the function is *called*. Example: --- const int f(void) { return 1; } int main() { return f(); } --- Output: --- $ gcc -Wextra y.c -o y y.c:1:1: warning: type qual

[Python-Dev] Re: [python-committers] [RELEASE] Python 3.11.0a3 is available

2021-12-09 Thread Victor Stinner
Yeah, another Python 3.11 alpha release! The asyncore, asynchat and smtpd modules are back into Python 3.11! They were removed but the removal has been reverted to respect the PEP 387 process: have two Python versions (3.10 and 3.11) emitting a DeprecationWarning. In general, I strongly advice yo

[Python-Dev] Re: subinterpreters and their possible impact on large extension projects

2021-12-15 Thread Victor Stinner
Hi Brett, IMO the PEP 630 is a good summary and a practical guide explaining how to port existing C extensions to newer C API which are compatible with subinterpreters, unloading a C extension and be able to load a C extension more than once (in the same interpreter): https://www.python.org/dev/pe

<    14   15   16   17   18   19   20   21   22   23   >