[Python-Dev] Exact date of Python 2 EOL?

2017-03-23 Thread Victor Stinner
Hi, I looked at the "Status of Python branches" to check if it was up to date. It's the case, thanks :-) But it recalled me that no exact date was decided for the official end of line of the the Python 2 branch (2.7 EOL). https://docs.python.org/devguide/#status-of-python-branches says January 1s

Re: [Python-Dev] Exact date of Python 2 EOL?

2017-03-23 Thread Victor Stinner
Le 23 mars 2017 19:56, "Terry Reedy" a écrit : > https://docs.python.org/devguide/#status-of-python-branches > says January 1st, 2020 Can we pick an official date? > The devguide list is effectively the official list. No one should plan on getting anything from pydev after 1/1/20. Oh sorry,

[Python-Dev] PyCharm debugger became 40x faster on Python 3.6 thanks to PEP 523

2017-03-25 Thread Victor Stinner
https://blog.jetbrains.com/pycharm/2017/03/inside-the-debugger-interview-with-elizaveta-shashkova/ "What changed in Python 3.6 to allow this? The new frame evaluation API was introduced to CPython in PEP 523 and it allows to specify a per-interpreter function pointer to handle the evaluation of f

Re: [Python-Dev] Not all public names in C API have the "Py" prefix

2017-03-27 Thread Victor Stinner
2017-03-27 12:22 GMT+02:00 Serhiy Storchaka : > Should we to do something with this? Maybe add Py-prefixed aliases and > temporary keep old names for compatibility (but allow to hide them if define > a special macro)? Is is possible to keep backward compatibility if an older version of the stable

[Python-Dev] Issue #21071: change struct.Struct.format type from bytes to str

2017-03-27 Thread Victor Stinner
Hi I would like to change struct.Struct.format type from bytes to str. I don't expect that anyone uses this attribute, and struct.Struct() constructor accepts both bytes and str. http://bugs.python.org/issue21071 It's just to be convenient: more functions accept str than bytes in Python 3. Examp

Re: [Python-Dev] Distutils frozen?

2017-03-27 Thread Victor Stinner
2017-03-27 19:32 GMT+02:00 Ryan Gonzalez : > Then I found bpo-11913 (https://bugs.python.org/issue11913), which said: > > This would be easy to fix, but as it would be considered a new feature, it > can’t go into distutils, which is frozen. Oh, that painful story. There was a huge "distutils2 proj

Re: [Python-Dev] I will be deleting the cpython-mirror repo on April 10

2017-03-27 Thread Victor Stinner
Oops, thanks for the reminder! I found two old pull requests that I forgot to rebase and republish on the new CPython Git repository. Victor 2017-03-27 22:13 GMT+02:00 Brett Cannon : > On the two-month anniversary of the GitHub migration I'm going to delete the > old git mirror: https://github.co

Re: [Python-Dev] OpenIndiana and Solaris support

2017-03-27 Thread Victor Stinner
2017-02-08 15:14 GMT+01:00 Jesus Cea : > On 08/02/17 11:24, Victor Stinner wrote: >> So I suggest to drop official Solaris support, but I don't propose to >> remove the C code specific to Solaris. In practice, I suggest to >> remove Solaris and OpenIndiana buildbots

[Python-Dev] Reminder: buildbots are alive :-)

2017-03-27 Thread Victor Stinner
Hi, Don't forget our brave buildbots which compile Python to run the full test suite, every day, every night, even if it's raining or worse! Python 3.7: http://buildbot.python.org/all/waterfall?category=3.x.stable&category=3.x.unstable Python 3.6: http://buildbot.python.org/all/waterfall?categor

[Python-Dev] Questions on the CPython Git master branch: how to exclude commits of 3.x branches?

2017-03-31 Thread Victor Stinner
Hi, The CPython repository was converted from Mercurial to Git. Before with Mercurial, we used extensively merges. For example, a bug was fixed in branche 3.5, merged into 3.6 and then merged into master. With the conversion to Git, some merges commit are removed, some others are kept. My questio

Re: [Python-Dev] Questions on the CPython Git master branch: how to exclude commits of 3.x branches?

2017-03-31 Thread Victor Stinner
2017-03-31 18:36 GMT+02:00 Ryan Gonzalez : > I think you want: > > git log --no-merges --first-parent Oh, I mised --first-parent: it seems like it fixed my issue, thanks! But --no-merges is not what I want. I want to see merge commits which are only in the master branch. Victor _

[Python-Dev] Fix or drop AIX buildbot?

2017-04-03 Thread Victor Stinner
Hi, We have a "PPC64 AIX 3.x" buildbot slave which fails on cloning the GitHub repository: "SSL certificate problem: unable to get local issuer certificate". It started to fail around Feb 11, 2017 (Build #294), probably when buildbots moved to GitHub, after CPython moved to GitHub. First build w

Re: [Python-Dev] Fix or drop AIX buildbot?

2017-04-03 Thread Victor Stinner
2017-04-03 16:00 GMT+02:00 David Edelsohn : > I have fixed the Git problem. I thought that it had been working. Thank you :-) I see tests running: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/567/steps/test/logs/stdio > The testsuite failures on AIX are issues with the AIX ke

Re: [Python-Dev] Fix or drop AIX buildbot?

2017-04-03 Thread Victor Stinner
I created http://bugs.python.org/issue29972 to skip tests known to fail on AIX. I already proposed two different pull requests which fix some tests (not all tests yet). Victor 2017-04-03 16:12 GMT+02:00 Victor Stinner : > 2017-04-03 16:00 GMT+02:00 David Edelsohn : >> I have fixe

[Python-Dev] CPython benchmark status, April 2017

2017-04-07 Thread Victor Stinner
Hi, I'm still working on analyzing past optimizations to guide future optimizations. I succeeded to identify multiple significant optimizations over the last 3 years. At least for me, some were unexpected like "Use the test suite for profile data" which made pickle 1.28x faster and pidigts 1.16x f

Re: [Python-Dev] CPython benchmark status, April 2017

2017-04-07 Thread Victor Stinner
2017-04-07 20:50 GMT+02:00 Terry Reedy : > The boxes are un-scaled. Which page? "Display All in a grid" has no scale, but if you select a single benchmark, there are scales. It's a number of seconds: "(less is better)" as written on the Y scale. > Is vertical always 'time for fixed work', so lowe

[Python-Dev] TLS issue (unknown CA) on test_imaplib

2017-04-27 Thread Victor Stinner
Hi, test_imaplib is starting to fail randomly causing CI tests to fail and buildbots to fail randomly: http://bugs.python.org/issue30175 Can somone please take a look to help me to fix this issue which becomes blocking? test.test_imaplib.RemoteIMAP_SSLTest.test_logincapa_with_client_certfile() f

Re: [Python-Dev] TLS issue (unknown CA) on test_imaplib

2017-04-27 Thread Victor Stinner
2017-04-27 15:41 GMT+02:00 Antoine Pitrou : > There's also a failure in test_nntplib: > https://travis-ci.org/python/cpython/jobs/226384811#L3157-L3171 That's a different issue, unrelated to the SSL issue. EOFError is probably as old as test_nntplib, there are 2 open issues to track this bug: ht

Re: [Python-Dev] TLS issue (unknown CA) on test_imaplib

2017-04-27 Thread Victor Stinner
2017-04-27 15:46 GMT+02:00 Victor Stinner : > 2017-04-27 15:41 GMT+02:00 Antoine Pitrou : >> There's also a failure in test_nntplib: >> https://travis-ci.org/python/cpython/jobs/226384811#L3157-L3171 > > That's a different issue, unrelated to the SSL issue. >

Re: [Python-Dev] TLS issue (unknown CA) on test_imaplib

2017-04-27 Thread Victor Stinner
2017-04-27 17:16 GMT+02:00 Victor Stinner : > Oh wait, it seems like nntplib now has a fail rate close to 100% :-/ I > created http://bugs.python.org/issue30188 and wrote a fix. I included > this nntplib fix in my imaplib PR... I have good news: * I pushed fixes for test_imaplib and tes

[Python-Dev] Status of Python buildbots

2017-05-03 Thread Victor Stinner
Hi, I spent last week working on fixing buildbots: https://www.python.org/dev/buildbot/ It should now be able again to rely on them to detect regressions. Changes: * Fix various bugs (I don't even recall which ones) * Fix multiple random failures * Fix dozen of warnings I also enhanced the

Re: [Python-Dev] Removing Python/thread_foobar.h

2017-05-03 Thread Victor Stinner
You can remove thread_foobar.h. I don't think that anyone still wants to use this template. The other thread_*.h files can be used as template as well. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/pyth

Re: [Python-Dev] Is adding support for os.PathLike an enhancement or bugfix?

2017-05-03 Thread Victor Stinner
If you start to backport support for the fspath protocol, be prepared to have to backport it *many* places. I expect that slowly in the near future, many functions will be patched to support the fspath protocol. I suggest to only do that in master. It's not that hard to cast manually to a string:

[Python-Dev] "make touch" replaced with "make regen-all"

2017-05-04 Thread Victor Stinner
s://github.com/python/cpython/commit/a5c62a8e9f0de6c4133825a5710984a3cd5e102b --- commit a5c62a8e9f0de6c4133825a5710984a3cd5e102b Author: Victor Stinner Date: Wed May 3 18:21:48 2017 +0200 bpo-23404: make touch becomes make regen-all (#1405) Don't rebuild generated files based on file modifi

Re: [Python-Dev] "make touch" replaced with "make regen-all"

2017-05-04 Thread Victor Stinner
Le 5 mai 2017 6:31 AM, "Nick Coghlan" a écrit : The note just needs to say that folks that care about doing "complete" builds need to adjust their command sequence to be "./configure && make regen-all && make install", rather than the previous pattern of "./configure && make install". Hum, yo

Re: [Python-Dev] "make touch" replaced with "make regen-all"

2017-05-05 Thread Victor Stinner
2017-05-05 6:31 GMT+02:00 Nick Coghlan : > For the benefit of Linux distros attempting to ensure they're doing > full "from source" builds, it would be good to note this in a "Notable > changes in maintenance releases", akin to the existing ones for 3.4 > and 2.7 (perhaps retitling the latter accor

[Python-Dev] Outdated GitHub clone of the old svn repository

2017-05-05 Thread Victor Stinner
Hi, Does someone know who owns the following Git clone of the old Subversion CPython repository? https://github.com/python-git/python/ I would suggest to remove it to avoid confusion. A friend pointed to me this repository and was surprised to see outdated code... Is https://github.com/python-gi

Re: [Python-Dev] Outdated GitHub clone of the old svn repository

2017-05-05 Thread Victor Stinner
2017-05-05 18:36 GMT+02:00 Jonathan Goble : > It appears to me to be an individual user rather than an organization. Oh nice, glad to meet you :-) So what do you think? Are you ok to remove this old clone? Or do you have reasons to keep it? Victor ___ P

Re: [Python-Dev] "make touch" replaced with "make regen-all"

2017-05-09 Thread Victor Stinner
2017-05-04 19:51 GMT+02:00 Raymond Hettinger : > Yes. It is perfectly reasonable to backport improvements to the tooling as > long as it doesn't break anyone's existing build process. I pushed my change to 2.7, 3.5, 3.6 and master (3.7) branches: "make" doesn't try to regenerate generated files

[Python-Dev] New buildbot-status mailing list to be notified on buildbot failures

2017-05-11 Thread Victor Stinner
Hi, You can now subscribe to this mailing list to get email notifications when the state of a buildbot changes from success (green)/warning (orange) to failure (red): https://mail.python.org/mm3/mailman3/lists/buildbot-status.python.org/ Only these state changes (green=>red, orange=>red) send an

Re: [Python-Dev] PEP 545: Python Documentation Translations

2017-05-21 Thread Victor Stinner
Champagne! Congrats *Julien*! Julien Palards is the one who really drives the effort to get a translated doc at python.org. Thanks also Naoki INADA for the help on that PEP. I didn't do much, except of pushing to get a PEP and get everything written down to make things going smoothly and making the

Re: [Python-Dev] Outdated GitHub clone of the old svn repository

2017-05-22 Thread Victor Stinner
3:23 PM, Guido van Rossum wrote: > >> OK I'll contact GitHub. >> >> On Fri, May 5, 2017 at 10:01 AM, Guido van Rossum >> wrote: >> >>> Let's coordinate who contacts GitHub. Victor, Brett or myself? >>> >>> On Fri, May 5, 2017 at

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

2017-05-23 Thread Victor Stinner
2017-05-22 13:17 GMT-05:00 Steve Dower : > Once the special protection is removed, most of these cases will become > OSError due to the general protection against segmentation faults. It didn't know that ctypes on Windows had a special protection against programming errors. I'm not aware of such p

[Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Victor Stinner
Hi, Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python 2.7? I can do the backport. https://docs.python.org/dev/library/ssl.html#ssl.MemoryBIO Cory Benfield told me that it's a blocking issue for him to implement his PEP 543 -- A Unified TLS API for Python 2.7: https://www

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Victor Stinner
sure that still applies and tests still > pass, I'd be a big fan. > > In addition to all the benefits you mentioned, it also substantially reduces > the diff between 2.7 and 3.x (or at least it did when I originally wrote > it). > > Cheers, > Alex > > On Tu

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Victor Stinner
2017-05-23 19:54 GMT-05:00 Cory Benfield : > In the absence of a Python 2.7 backport, Requests is required to basically > use the same solution that Twisted currently does: namely, a mandatory > dependency on PyOpenSSL. Last time I looked at requests, it embedded all its dependencies. I don't li

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Victor Stinner
Le 23 mai 2017 19:46, "Nick Coghlan" a écrit : (...) to the RHEL system Python by switching the default behaviour for new installs to be to verify SSL/TLS certificates against the system trust store Awesome! Security, I see security everywhere! Stay safe, Victor ___

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Victor Stinner
Le 23 mai 2017 17:49, "Alex Gaynor" a écrit : I'm +1 on this, I even wrote the patch: https://bugs.python.org/issue22559 :-) If you're interested in making sure that still applies and tests still pass, I'd be a big fan. Oh sorry, I wasn't aware of that one. Sure, will do. Victor __

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-23 Thread Victor Stinner
Le 23 mai 2017 20:43, "David Wilson" a écrit : In which case, what is to prevent Requests from just depending on pyOpenSSL as usual? >From what I heard, pyOpenSSL development is slowing down, so I'm not sure that it's really safe and future-proof (TLS 1.3 anyone?). I'm still writing 2.7 code e

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

2017-05-23 Thread Victor Stinner
Sure, make your change and then update libffi! Victor Le 23 mai 2017 18:19, "Steve Dower" a écrit : > On 23May2017 1212, Victor Stinner wrote: > >> 2017-05-22 13:17 GMT-05:00 Steve Dower : >> >>> Once the special protection is removed, most of these cas

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

2017-05-25 Thread Victor Stinner
Hi Ben, I am not convinced that combining operations will have a significant impact in term of performance. Mark Shanon implemented that in his HotPy project. I proposed a RETURN_NONE opcode to combine LOAD_CONST with RETURN_VALUE. The issue was rejected because I failed to show any speedup. htt

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-25 Thread Victor Stinner
Le 25 mai 2017 1:26 PM, "Antoine Pitrou" a écrit : System admins can add the company CA at the system level in the system's CA cert store, they have no need for a Python API. If I understood correctly, since the Python ssl module is unable to load system CAs (at least on Python 2.7) on Windows

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

2017-05-25 Thread Victor Stinner
After reading Cesare Di Mauro's email, I realized that I was thinking to WPython in fact... Victor Le 25 mai 2017 8:11 PM, "Mark Shannon" a écrit : > > > On 25/05/17 03:47, Victor Stinner wrote: > >> Hi Ben, >> >> I am not convinced that combining

Re: [Python-Dev] "Micro-optimisations can speed up CPython"

2017-05-29 Thread Victor Stinner
Hi, Thank you for the links. -PyObject *value = GETLOCAL(oparg); +PyObject *value = GETLOCAL((unsigned)oparg); Oh, I remember that I proposed to change oparg type to unsigned when Demur wrote the WORDCODE change. I even wrote a patch, but I got a segfault and was unable t

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

2017-05-30 Thread Victor Stinner
I wrote a first PEP draft: https://github.com/python/peps/pull/272 Victor 2017-05-24 2:46 GMT+02:00 Victor Stinner : > Hi, > > Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python > 2.7? I can do the backport. > > https://docs.python.org/dev/library/ssl.

[Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-05-31 Thread Victor Stinner
: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7 Version: $Revision$ Last-Modified: $Date$ Author: Victor Stinner , Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 30-May-2017 Abstract Backport the ssl.MemoryBIO and ssl.SSLObject classes from Python 3 to Python 2.7 to e

[Python-Dev] PEP 7 and braces { .... } on if

2017-05-31 Thread Victor Stinner
Hi, I have a question on the CPython coding code for C code, the PEP 7: https://www.python.org/dev/peps/pep-0007/ """ Code structure: (...); braces are strongly preferred but may be omitted where C permits, and they should be formatted as shown: if (mro != NULL) { ... } else { ... } """

Re: [Python-Dev] PEP 7 and braces { .... } on if

2017-05-31 Thread Victor Stinner
Previous discussion which added "strongly preferred" to the PEP 7, January 2016: https://mail.python.org/pipermail/python-dev/2016-January/142746.html Victor 2017-05-31 16:11 GMT+02:00 Victor Stinner : > Hi, > > I have a question on the CPython coding code for C code,

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-05-31 Thread Victor Stinner
2017-05-31 17:45 GMT+02:00 Jim Baker : > Given that this proposed new feature is for 2.7 to support event loop usage > and not a security fix, I'm -1 on this change. In particular, it runs > counter to the justification policy stated in PEP 466. Hum, it seems like the PEP 546 abstract is incomplet

Re: [Python-Dev] PEP 7 and braces { .... } on if

2017-06-01 Thread Victor Stinner
2017-05-31 19:27 GMT+02:00 Guido van Rossum : > I interpret the PEP (...) Right, the phrasing requires to "interpret" it :-) > (...) as saying that you should use braces everywhere but not > to add them in code that you're not modifying otherwise. (I.e. don't go on a > brace-adding rampage.) If a

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Victor Stinner
2017-06-01 9:38 GMT+02:00 Larry Hastings : > When CPython's small block allocator was first merged in late February 2001, > it allocated memory in gigantic chunks it called "arenas". These arenas > were a massive 256 KILOBYTES apiece. The arena size defines the strict minimum memory usage of Pyth

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Victor Stinner
2017-06-01 10:19 GMT+02:00 Antoine Pitrou : > Yes, this is the same kind of reason the default page size is still 4KB > on many platforms today, despite typical memory size having grown by a > huge amount. Apart from the cost of fragmentation as you mentioned, > another issue is when many small Py

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Victor Stinner
2017-06-01 10:23 GMT+02:00 INADA Naoki : > AFAIK, allocating arena doesn't eat real (physical) memory. > > * On Windows, VirtualAlloc is used for arena. Real memory page is assigned > when the page is used first time. > * On Linux and some other *nix, anonymous mmap is used. Real page is > as

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Victor Stinner
2017-06-01 10:41 GMT+02:00 Larry Hastings : > On 06/01/2017 01:19 AM, Antoine Pitrou wrote: > If you'd like to go that way anyway, I would suggest 1MB as a starting > point in 3.7. > > I understand the desire for caution. But I was hoping maybe we could > experiment with 4mb in trunk for a while?

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Victor Stinner
2017-06-01 10:57 GMT+02:00 Antoine Pitrou : > If Requests is to remain 2.7-compatible, it's up to Requests to do the > necessary work to do so. In practice, CPython does include Requests in ensurepip. Because of that, it means that Requests cannot use any C extension. CPython 2.7 ensurepip prevent

Re: [Python-Dev] "Global freepool"

2017-06-01 Thread Victor Stinner
2017-06-01 10:40 GMT+02:00 Antoine Pitrou : > This is already exactly how PyObject_Malloc() works. (...) Oh ok, good to know... > IMHO the main thing the > private freelists have is that they're *private* precisely, so they can > avoid a couple of conditional branches. I would like to understand

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-01 Thread Victor Stinner
It seems very complex and not portable at all to "free" a part of an arena. We already support freeing a whole arena using munmap(). I was a huge enhancement in our memory allocator. Change made in Python 2.5? I don't recall, ask Evan Jones: http://www.evanjones.ca/memoryallocator/ :-) I'm not sur

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Victor Stinner
2017-06-01 18:51 GMT+02:00 Chris Angelico : > Hmm. So it's really hard to know. Pity. I suppose it's too much to ask > for IP-based stat exclusion for the most commonly-used CI systems > (Travis, Circle, etc)? Still, it does look like most pip usage happens > on Linux. Also, it seems likely that th

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Victor Stinner
2017-06-01 19:09 GMT+02:00 Barry Warsaw : > By 2020, only Ubuntu 16.04 and 18.04 will still be maintained, so while 18.04 > will likely contain whatever the latest 2.7 is available at that time, 16.04 > won't track upstream point releases, but instead will get select cherry > picks. For good reaso

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-01 Thread Victor Stinner
2017-06-01 19:10 GMT+02:00 Nathaniel Smith : > (...) since pip/requests/everyone else pushing for > this only want to use ssl.MemoryBIO on Linux. Their plan on Windows/MacOS > (IIUC) is to stop using the ssl module entirely in favor of new ctypes > bindings for their respective native TLS libraries

Re: [Python-Dev] "Global freepool"

2017-06-01 Thread Victor Stinner
2017-06-01 22:16 GMT+02:00 Serhiy Storchaka : > The issue [1] still is open. Patches neither applied nor rejected. They > exposes the speed up in microbenchmarks, but it is not large. Up to 40% for > iterating over enumerate() and 5-7% for hard integer computations like > base85 encoding or spectra

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Victor Stinner
Thanks Cory for the long explanation. Let me try to summarize (tell me if I'm wrong). We have 3 options: * Do nothing: reject the PEP 546 and let each project handles security on its own (current status co) * Write *new* C code, maybe using certitude as a starting point, to offload certifcate val

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Victor Stinner
I would be curious of another test: use pymalloc for objects larger than 512 bytes. For example, allocate up to 4 KB? In the past, we already changed the maximum size from 256 to 512 to support most common Python objects on 64-bit platforms. Since Python objects contain many pointers: switching fr

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-07 Thread Victor Stinner
2017-06-07 10:56 GMT+02:00 Nathaniel Smith : > Another testing challenge is that the stdlib ssl module has no way to > trigger a renegotiation, and therefore there's no way to write tests > to check that it properly handles a renegotiation, even though > renegotiation is by far the trickiest part o

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-08 Thread Victor Stinner
2017-06-08 10:30 GMT+02:00 Cory Benfield : > This is what I was worried about. Moving to require PyOpenSSL *also* locks > us out of Jython support, at least for the time being. That’s another point > in the “con” column for making PyOpenSSL a mandatory dependency. Even if we do backport MemoryBIO

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-08 Thread Victor Stinner
applications. I should take time to read the last messages in this thread and try to summarize them in the PEP ;-) Victor 2017-06-08 13:06 GMT+02:00 Donald Stufft : > > On Jun 8, 2017, at 6:36 AM, Victor Stinner wrote: > > 2017-06-08 10:30 GMT+02:00 Cory Benfield : > > This is

[Python-Dev] Buildbot report, june 2017

2017-06-08 Thread Victor Stinner
Hi, With the help of Zachary Ware, we succeed to set up a mailing list getting email notifications when a buildbot fails whereas it previously succeeded (green/success or orange/warnings): https://mail.python.org/mm3/mailman3/lists/buildbot-status.python.org/ Please subscribe if you want to stay

Re: [Python-Dev] [python-committers] IMPORTANT: Python 3.6.2 Maintenance Release Release Candidate in 3+ days (Monday 2017-06-12 12:00 UTC)

2017-06-09 Thread Victor Stinner
Oh, about very annoying 3.6 bug, there was a regression caused by FASTCALL optimizations. It's now fixed in the 3.6 branch: https://github.com/python/cpython/commit/f0ff849adc6b4a01f9d1f08d9ad0f1511ff84541 Victor 2017-06-09 5:34 GMT+02:00 Ned Deily : > We are approaching the end of the second c

[Python-Dev] Remove embedded expat library?

2017-06-09 Thread Victor Stinner
Hi, Python embeds a copy of the expat library which already got two major security vulnerabilities: "CVE-2016-0718: expat bug #537" http://python-security.readthedocs.io/vuln/cve-2016-0718_expat_bug_537.html "Issue #26556: Expat 2.1.1" http://python-security.readthedocs.io/vuln/issue_26556_expat

Re: [Python-Dev] On "PEP 546 — Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7"

2017-06-10 Thread Victor Stinner
> On Fri, Jun 9, 2017 at 4:56 PM, Benjamin Peterson > wrote: > Therefore, as 2.7 release manager, I'm accepting the PEP. 2017-06-10 3:03 GMT+02:00 Guido van Rossum : > Well reasoned! Guido: by default, you are the only one who pronounces officially on a PEP. Should I understand that you approved

Re: [Python-Dev] On "PEP 546 — Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7"

2017-06-10 Thread Victor Stinner
Thank you Benjamin ;-) Victor Le 10 juin 2017 01:58, "Benjamin Peterson" a écrit : > The reason we're having this conversation at all is probably a matter of > timing. If MemoryBIO was in Python 3 when PEP 466 was accepted, it surely > would have come along for the ride to 2.7. I believe PEP 46

Re: [Python-Dev] On "PEP 546 — Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7"

2017-06-10 Thread Victor Stinner
Le 10 juin 2017 22:09, "Guido van Rossum" a écrit : Let's retroactively make Benjamin the BDFL-delegate for this PEP. The effect is the same: the PEP is officially accepted. Ok fine, I will update the PEP and then start to work on review the old implementation written by Alex Gaynor. Victor __

Re: [Python-Dev] Remove embedded expat library?

2017-06-11 Thread Victor Stinner
Le 11 juin 2017 09:38, "Ronald Oussoren" a écrit : I don’t think it would be a good idea to rely on the system provided libexpat on macOS, as Apple is not exactly fast w.r.t. upgrading their external dependencies and could easily stop updating libraries when the no longer need them (see for examp

[Python-Dev] PEP 538 warning at startup: please remove it

2017-06-12 Thread Victor Stinner
Hi, Nick Coghlan pushed his implementation of his PEP 538: nice! Nice step forward to UTF-8 everywhere ;-) I would prefer to not be annoyed by warning messages about encodings at startup if possible: "Python detected LC_CTYPE=C: LC_CTYPE coerced to C.UTF-8 (set another locale or PYTHONCOERCECLOC

Re: [Python-Dev] PEP 538 warning at startup: please remove it

2017-06-12 Thread Victor Stinner
2017-06-12 11:35 GMT+02:00 INADA Naoki : > I think "Good practice" is set `LC_CTYPE=C.UTF-8` environment variable, > as warning says. I like using LANG=C to display a manual page in english. Technically, I know that it's wrong, but it works. I don't see the point of the warning. I'm able to render

Re: [Python-Dev] PEP 538 warning at startup: please remove it

2017-06-12 Thread Victor Stinner
2017-06-12 15:28 GMT+02:00 INADA Naoki : >> I like using LANG=C to display a manual page in english. > > Me too. But we can use LC_CTYPE=C.UTF-8 with LANG=C. My point is that LANG=C is easy to remember and "it just works". > And LC_CTYPE=C.UTF-8 is much easier (and ideal) than > PYTHONLOCALECOER

Re: [Python-Dev] On "PEP 546 — Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7"

2017-06-12 Thread Victor Stinner
FYI I started to work on the implementation: I rebased Alex Gaynor's patch written in 2014 and converted it to a pull request. http://bugs.python.org/issue22559 Victor 2017-06-10 1:56 GMT+02:00 Benjamin Peterson : > The reason we're having this conversation at all is probably a matter of > timing

Re: [Python-Dev] PEP 538 warning at startup: please remove it

2017-06-13 Thread Victor Stinner
ost recent call last):', ? ^ + [b'Traceback (most recent call last):', ? ^ b' File "", line 1, in ', b'DeprecationWarning: Message'] Victor 2017-06-12 10:56 GMT+02:00 Victor Stinner : > Hi, > > Nick Coghlan pushed his implementation o

Re: [Python-Dev] PEP 538 warning at startup: please remove it

2017-06-13 Thread Victor Stinner
2017-06-13 8:51 GMT+02:00 Nick Coghlan : > If we turned the warning off by default, but retained an easy > "on-switch" for the warning to help track places where locale coercion > is triggering unexpectedly (e.g. "PYTHONCOERCECLOCALE=warn"), that > would probably give us the best of both worlds. H

Re: [Python-Dev] Issue #21071: change struct.Struct.format type from bytes to str

2017-06-16 Thread Victor Stinner
2017-06-16 10:40 GMT+02:00 Nick Coghlan : > As long as it's noted in the "Porting to Python 3.7" section of the > 3.7 What's New guide, this seems like a sensible change to me. Yes, the change is already documented there: https://github.com/python/cpython/pull/845/files Victor ___

[Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-16 Thread Victor Stinner
Hi, Last weeks, I worked on a new tool to bisect failing tests because it's painful to bisect manually reference leaks (I remove as much code as possible until the code is small enough to be reviewable manually). See the bisect_test.py script attached to this issue: http://bugs.python.org/issue29

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-16 Thread Victor Stinner
t the script is stuck and you consider that the output is small enough to a manual review. Victor > On Fri, 16 Jun 2017 at 09:06 Victor Stinner > wrote: >> >> Hi, >> >> Last weeks, I worked on a new tool to bisect failing tests because >> it's painful to b

Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)

2017-06-20 Thread Victor Stinner
Hi, Re: "[Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)" 2017-02-24 5:36 GMT+01:00 Steven D'Aprano : > I am not qualified to judge the merits of this, but it does seem > worrying that (alledgedly) the Python security team hasn't responded for > over 12 months

Re: [Python-Dev] Python FTP Injections Allow for Firewall Bypass (oss-security advisory)

2017-06-20 Thread Victor Stinner
istian > Heimes. I think that was a dark year for the PSRT. > > On Tue, Jun 20, 2017 at 3:35 PM, Victor Stinner > wrote: >> >> Hi, >> >> Re: "[Python-Dev] Python FTP Injections Allow for Firewall Bypass >> (oss-security advisory)" >> >&g

[Python-Dev] test_sax and test_random fail on Python 3.6.2rc1 on Windows

2017-06-21 Thread Victor Stinner
Hi, The end-of-line hell is not over, test_sax and test_random tests are still failing if you install Python 3.6.2rc1 on Windows: http://bugs.python.org/issue27425#msg296519 These tests rely on files in Lib/test/. The end-of-line of these files is controlled by .gitattributes, but it seems like t

Re: [Python-Dev] test_sax and test_random fail on Python 3.6.2rc1 on Windows

2017-06-21 Thread Victor Stinner
2017-06-21 16:10 GMT+02:00 Steve Dower : > Do we have a minimum git version requirement? Maybe I need to update my > build machine. After we added .gitattributes, we had to force a fresh Git checkout on buildbots. Otherwise, they kept the old and wrong end of line. Maybe try to move my checkout an

Re: [Python-Dev] test_sax and test_random fail on Python 3.6.2rc1 on Windows

2017-06-21 Thread Victor Stinner
> (Or the fact that it took me three goes to get to that point – my least > favourite part of the git migration is git...) > > > > When I get to work I’ll clean it up and try again to see if it repros or was > a random occurrence. > > > > Top-posted from my Windows ph

Re: [Python-Dev] [python-committers] Proposed release schedule for Python 3.5.4

2017-06-22 Thread Victor Stinner
For 3.4, please review my pending security fixes :-) There are more of them. About the cipher list in ssl, the change itself is simple but it's to blacklist DES and 3DES since it has been proved that these ciphers are really too weak nowadays: http://python-security.readthedocs.io/vuln/cve-2016-21

Re: [Python-Dev] [python-committers] Proposed release schedule for Python 3.5.4

2017-06-23 Thread Victor Stinner
2017-06-22 17:56 GMT+02:00 Brett Cannon : > On Thu, 22 Jun 2017 at 02:32 Larry Hastings wrote: >> Seriously, though, I was mostly hoping other people would handle the >> security stuff and just keep me informed. If I'm the only one permitted to >> accept PRs into 3.4 (and soon 3.5), okay, I can w

Re: [Python-Dev] [python-committers] Proposed release schedule for Python 3.5.4

2017-06-23 Thread Victor Stinner
2017-06-23 15:19 GMT+02:00 Larry Hastings : > Do you need write access to the branch in order to get Travis CI working? As soon as someone reviews my proposed 3.4 patches, no :-) I will work on a PR. Victor ___ Python-Dev mailing list Python-Dev@python.

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Victor Stinner
Hi, I updated my bisect_test.py script to simplify its command line: https://github.com/haypo/misc/blob/master/python/bisect_test.py To debug a reference leak in test_os, now just type: $ ./python bisect_test.py -R 3:3 test_os Example of output: --- (...) Failing tests (1): * test.test_o

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-26 Thread Victor Stinner
2017-06-26 21:58 GMT+02:00 Brett Cannon : > I don't see why regrtest isn't the right place for this. The current regrest CLI isn't designed for subcommands, and I don't want to "pollute" regrtest with multiple options for bisect. Currently, my script has already 4 options: haypo@selma$ python3 ~/

Re: [Python-Dev] Helping contributors with chores (do we have to?)

2017-06-27 Thread Victor Stinner
Small enhancement: I added Python 3.5 support to blurb with the help of Serhiy Storchaka ;-) Victor 2017-06-25 10:33 GMT+02:00 Antoine Pitrou : > On Sat, 24 Jun 2017 21:37:46 -0700 > Larry Hastings wrote: >> On 06/24/2017 09:14 PM, Serhiy Storchaka wrote: >> > Not only core developers make PRs f

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-27 Thread Victor Stinner
2017-06-27 7:33 GMT+02:00 Serhiy Storchaka : > You could make it just a submodule in the test package. > > ./python -m test.bisect -R 3:3 test_os I like the idea :-) I proposed a PR which was approved by Yury Selivanov, and so I just merged it! It means that you can now play with "./python -m

Re: [Python-Dev] Micro-optimizations by adding special-case bytecodes?

2017-06-28 Thread Victor Stinner
(Victor wears his benchmark hat.) 2017-06-28 15:50 GMT+02:00 Ben Hoyt : > ../test_none.sh > x = 1234; x is None -- 2000 loops, best of 5: 19.8 nsec per loop > x = 1234; x is not None -- 1000 loops, best of 5: 20 nsec per loop > x = None; x is None -- 1000 loops, best of 5: 20.7 nsec pe

[Python-Dev] Buildbot report (almost July)

2017-06-29 Thread Victor Stinner
Hi, Our buildbots are now even more stable than in my previous buildbot report. Many random failures have been fixed, even if there are still many rare random failures (most of them are related to multiprocessing). Search for issues created by "haypo" (me!) with the Component "Tests" to see all

Re: [Python-Dev] Buildbot report (almost July)

2017-06-29 Thread Victor Stinner
2017-06-29 19:13 GMT+02:00 Terry Reedy : > How about compiler warnings (and errors)? That would be nice to have, but I don't have the bandwidth to handle this goal. > When I compile on Windows, there > are a boatload of orange-yellow warnings. Some are about using a deprecated > featured; some

[Python-Dev] (no subject)

2017-06-30 Thread Victor Stinner
Hi Robb, 2017-06-29 23:34 GMT+02:00 Rob Boehne : > I¹m new to the list, and contributing to Python specifically, and I¹m > interested in getting master and 3.6 branches building and working > ³better² on UNIX. > I¹ve been looking at a problem building 3.6 on HP-UX and see a PR was > merged into ma

Re: [Python-Dev] Bug in buildbot view (changes column)

2017-06-30 Thread Victor Stinner
FYI there is an ongoing migration from the old patched Buildbot 0.8 to a vanilla Buildbot 0.9: https://github.com/python/buildmaster-config/pull/12 Sorry, I don't know much more. There are private discussion between 2 buildbot developers, Zachary Ware and me, but Zach is in vacation :-) We will t

Re: [Python-Dev] New work-in-progress bisection tool for the Python test suite (in regrtest)

2017-06-30 Thread Victor Stinner
eted in 14 iterations and 0:00:14 --- So the bisection also works to find a failing test method when running a test file twice ;-) Victor 2017-06-28 2:39 GMT+02:00 Victor Stinner : > 2017-06-27 7:33 GMT+02:00 Serhiy Storchaka : >> You could make it just a submodule in the test package. &g

<    13   14   15   16   17   18   19   20   21   22   >