[Python-Dev] stat module in C -- what to do with stat.py?

2013-06-20 Thread Christian Heimes
Hello, I have re-implemented the entire stat module in C. [1] It's a necessary step to fix portability issues. For most constants POSIX standards dictate only the name of the constant and its meaning but not its value. Only the values of permission bits (0644 == rw-r--r--) have fixed values. For

Re: [Python-Dev] stat module in C -- what to do with stat.py?

2013-06-20 Thread Christian Heimes
Am 20.06.2013 15:21, schrieb Florent: > we already have "_pyio.py", we could have "_pystat.py". Works for me. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailm

Re: [Python-Dev] stat module in C -- what to do with stat.py?

2013-06-20 Thread Christian Heimes
Am 20.06.2013 17:35, schrieb Benjamin Peterson: > 2013/6/20 Charles-François Natali : >> 2013/6/20 Thomas Wouters : >>> If the .py file is going to be wrong or incomplete, why would we want to >>> keep it -- or use it as fallback -- at all? If we're dead set on having a >>> .py file instead of requ

Re: [Python-Dev] stat module in C -- what to do with stat.py?

2013-06-21 Thread Christian Heimes
Am 21.06.2013 13:45, schrieb Antoine Pitrou: > For me, PEP 399 should not be considered a requirement but a guideline. > stat.py is algorithmically trivial and I don't think it saves much work > for authors of third-party Python implementations. The module's content is rather boring. It's just a b

Re: [Python-Dev] stat module in C -- what to do with stat.py?

2013-06-21 Thread Christian Heimes
Am 21.06.2013 17:47, schrieb Benjamin Peterson: > > Kill it. I would argue having incorrect constants makes the > implementation incompatible with CPython anyway. This not that much > work (as long as there are tests that the constants exist at least) to > emulate. > My patch adds extensive test

Re: [Python-Dev] cpython: Fix memory leak in pyexpat PyUnknownEncodingHandler

2013-06-29 Thread Christian Heimes
Am 29.06.2013 21:29, schrieb R. David Murray: > I don't know if this is the commit at fault or not, but we are seeing > segfaults in test_xml_etree on the buildbots now. Yeah, it's my fault. Thanks! >> u = PyUnicode_Decode((char*) template_buffer, 256, name, "replace"); >> -if (u == NULL

Re: [Python-Dev] lament for the demise of unbound methods

2013-07-04 Thread Christian Heimes
Am 04.07.2013 13:21, schrieb Chris Withers: > There doesn't appear to be any way in Python 3 to do this, which is a > little surprising and frustrating... > > What am I missing here? I removed unbound methods almost six years ago: http://hg.python.org/cpython/rev/48af6375207e Christian

[Python-Dev] Python 3.4 and Windows XP: just 45 days until EOL

2013-07-11 Thread Christian Heimes
Hi, how do you feel about dropping Windows XP support for Python 3.4? It would enable us to use some features that are only available on Windows Vista and newer, for example http://bugs.python.org/issue6926 and http://bugs.python.org/issue1763 . PEP 11 says: A new feature release X.Y.0 will sup

Re: [Python-Dev] Python 3.4 and Windows XP: just 45 days until EOL

2013-07-12 Thread Christian Heimes
Am 12.07.2013 03:49, schrieb Ben Hoyt: > I guess it has to be dropped at some stage, but with Windows XP it's a > case of "XP is dead. Long live XP!" There are still an awful lot of XP > boxes out there, and I'd kind hate to see support dropped completely. We > still use it here at home. > > Wikip

Re: [Python-Dev] cpython: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure

2013-07-16 Thread Christian Heimes
Am 17.07.2013 00:03, schrieb Victor Stinner: > 2013/7/16 Serhiy Storchaka : >>> http://hg.python.org/cpython/rev/533eb9ab895a >>> summary: >>>Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure >>> >>> bytes is NULL on _PyBytes_Resize() failure >> >> Why not Py_DECREF? > > Because Py_

Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.

2013-07-20 Thread Christian Heimes
> This will wrong when strlen(fname) is 30. strncat() will copy only 30 > bytes, without terminal NUL. That's not how strncat() works. strncat(dest, src, n) writes n+1 chars to the end of dest: n chars from src and +1 for the final NUL char. For this reason dest must be large enough to hold strlen

Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.

2013-07-20 Thread Christian Heimes
Am 20.07.2013 14:23, schrieb Serhiy Storchaka: > This will wrong when strlen(fname) is 30. strncat() will copy only 30 > bytes, without terminal NUL. http://linux.die.net/man/3/strncat The strncat() function is similar, except that * it will use at most n bytes from src; and * src does not need

Re: [Python-Dev] cpython: Use strncat() instead of strcat() to silence some warnings.

2013-07-20 Thread Christian Heimes
Am 20.07.2013 15:56, schrieb Serhiy Storchaka: > Oh, true. strncat() always results NUL-terminated string. It's strncpy() > can produce not NUL-terminated string. Sorry for noise. You're welcome! Better safe than sorry. :) ___ Python-Dev mailing list P

Re: [Python-Dev] cpython (3.3): return NULL here

2013-07-23 Thread Christian Heimes
Am 23.07.2013 07:08, schrieb benjamin.peterson: > http://hg.python.org/cpython/rev/042ff9325c5e > changeset: 84804:042ff9325c5e > branch: 3.3 > parent: 84789:bb63f813a00f > user:Benjamin Peterson > date:Mon Jul 22 22:08:09 2013 -0700 > summary: > return NULL here > >

Re: [Python-Dev] cpython (3.3): return NULL here

2013-07-23 Thread Christian Heimes
Am 23.07.2013 17:10, schrieb Benjamin Peterson: >> PyErr_SetFromErrno() already and always returns NULL. Or do you prefer >> to return NULL explicitly? > > It might always return NULL, but the compiler sees (PyObject *)NULL > when this function returns dl_funcptr. Oh, you are right. I must have m

Re: [Python-Dev] cpython: Add modeling file for Coverity Scan.

2013-07-23 Thread Christian Heimes
Am 23.07.2013 08:27, schrieb Antoine Pitrou: > On Tue, 23 Jul 2013 01:31:24 +0200 (CEST) > christian.heimes wrote: >> + >> +typedef int sdigit; >> +typedef long Py_ssize_t; > > Can't you write "typedef ssize_t Py_ssize_t" instead? No, but it really doesn't matter. Coverity just needs a similar t

Re: [Python-Dev] Official github mirror for CPython?

2013-07-25 Thread Christian Heimes
Am 25.07.2013 16:29, schrieb Eli Bendersky: > Hi all, > > I've been looking for a Github mirror for Python, and found two: > > * https://github.com/python-git/python has a lot of forks/watches/starts > but seems to be very out of date (last updated 4 years ago) > * https://github.com/python-mirro

Re: [Python-Dev] Official github mirror for CPython?

2013-07-25 Thread Christian Heimes
Am 25.07.2013 16:48, schrieb Brian Curtin: > On Thu, Jul 25, 2013 at 9:37 AM, Christian Heimes > wrote: >> Am 25.07.2013 16:29, schrieb Eli Bendersky: >>> Hi all, >>> >>> I've been looking for a Github mirror for Python, and found two: >>> &g

[Python-Dev] Coverity Scan

2013-07-25 Thread Christian Heimes
Hello, this is an update on my work and the current status of Coverity Scan. Maybe you have noticed a checkins made be me that end with the line "CID #". These are checkins that fix an issue that was discovered by the static code analyzer Coverity. Coverity is a commercial product but it's a free

Re: [Python-Dev] Coverity Scan

2013-07-25 Thread Christian Heimes
Am 26.07.2013 00:32, schrieb Terry Reedy: > I found the answer here > https://docs.google.com/file/d/0B5wQCOK_TiRiMWVqQ0xPaDEzbkU/edit > Coverity Integrity Level 1 is 1 (defect/1000 lines) > Level 2 is .1 (we have passed that) > Level 3 is .01 + no major defects + <20% (all all defects?) false > po

Re: [Python-Dev] Coverity Scan

2013-07-25 Thread Christian Heimes
Am 26.07.2013 00:50, schrieb Antoine Pitrou: > "Excellence"? The term is too weak, I would say "perfection" at least, > but perhaps we should go as far as "divinity". Don't forget that Python can offer lots of places to keep your bike clean and dry ... *scnr* __

Re: [Python-Dev] Coverity Scan

2013-07-25 Thread Christian Heimes
Am 26.07.2013 00:00, schrieb Terry Reedy: >> http://www.coverity.com/company/press-releases/read/coverity-introduces-monthly-spotlight-series-for-coverity-scan-open-source-projects >> > > The intention is to promote the best of open source to industry. I think it's also a marketing tool. They lik

Re: [Python-Dev] Coverity Scan

2013-07-26 Thread Christian Heimes
Am 26.07.2013 14:56, schrieb Eli Bendersky: > Just a quick question - is there a chance to convince Coverity to detect > Python refcounting leaks in C API code :-) ? This could be useful not > only for Python but for extensions too. As it stands now, Coverity's > leak detection is Python must be pr

Re: [Python-Dev] Coverity Scan

2013-07-26 Thread Christian Heimes
Am 26.07.2013 16:29, schrieb Brett Cannon: > Have you tried asking for help from Coverity? They have been rather nice > so far and they may be willing to just give us free help in getting the > modeling file set up properly. Yes, I'm in contact with Dakshesh. I was able to figure out one model for

[Python-Dev] C code coverage report with lcov

2013-07-29 Thread Christian Heimes
Hi, I have done some experiments with GCC's gcov and lcov to get the C code coverage of our unit test suite. You may find today's report at http://tiran.bitbucket.org/python-lcov/ I'm working on a patch for our Makefile to include all steps in one simple make tag. http://bugs.python.org/issue1

Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Christian Heimes
Am 29.07.2013 19:58, schrieb Brett Cannon: > I took a quick poke around and it seems some things are legitimately not > being executed, while others are error conditions that we wouldn't > expect to occur (e.g. memory exhaustion). If we ever decide to get > serious about code coverage (both C and P

Re: [Python-Dev] C code coverage report with lcov

2013-07-29 Thread Christian Heimes
Am 29.07.2013 21:38, schrieb Brett Cannon: > If there's a way to report just function coverage then I think > that's a great place to start. lcov's genhtml command doesn't support just function coverage. But I have removed branch coverage. It makes the report a little bit more readable. Christian

Re: [Python-Dev] C code coverage report with lcov

2013-07-30 Thread Christian Heimes
Am 29.07.2013 19:15, schrieb Christian Heimes: > Hi, > > I have done some experiments with GCC's gcov and lcov to get the C code > coverage of our unit test suite. You may find today's report at > > http://tiran.bitbucket.org/python-lcov/ > > I'm working

Re: [Python-Dev] C code coverage report with lcov

2013-07-30 Thread Christian Heimes
Am 29.07.2013 21:48, schrieb Antoine Pitrou: > Ideally, we should run coverage runs on different systems (Unices, > Windows...) and merge the results together, so that we know which paths > are really uncovered. I don't that is easily possible. The coverage report depends on GCC and its gcov exten

Re: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Silence warning about set but unused variable inside compile_atom() in

2013-08-01 Thread Christian Heimes
Am 01.08.2013 09:03, schrieb Ronald Oussoren: > > On 31 Jul, 2013, at 23:50, christian.heimes > wrote: > >> http://hg.python.org/cpython/rev/0e09588a3bc2 >> changeset: 84939:0e09588a3bc2 >> parent: 84937:809a64ecd5f1 >> parent: 84938:83a55ca93

[Python-Dev] SSL issues in Python stdlib and 3rd party code

2013-08-12 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hello, last week Ryan Sleevi of the Google Chrome Security Team has informed us about about two issues in Python's SSL module. I already new about the cause of the first bug and suspected that our SSL module suffers from the second bug but I was una

Re: [Python-Dev] SSL issues in Python stdlib and 3rd party code

2013-08-13 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 CVE-2013-4238 has been signed to NULL bytes in subjectAltName issue. http://bugs.python.org/issue18709 http://www.openwall.com/lists/oss-security/2013/08/13/2 Should we assign a CVE to issue in ssl.match_hostname(), too? Even more projects have

[Python-Dev] PEP 452 API for Cryptographic Hash Functions v2.0

2013-08-15 Thread Christian Heimes
hling , Christian Heimes Status: Draft Type: Informational Created: 15-Aug-2013 Post-History: Replaces: 247 Abstract There are several different modules available that implement cryptographic hashing algorithms such as MD5 or SHA. This document specifies a standard API for such algor

[Python-Dev] Coverity Scan Spotlight Python

2013-08-29 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hello, Coverity has published its "Coverity Scan Spotlight Python" a couple of hours ago. It features a summary of Python's ecosystem, an interview with me about Python core development and a defect report. The report is awesome. We have reached a d

Re: [Python-Dev] Coverity Scan Spotlight Python

2013-08-29 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am 30.08.2013 00:46, schrieb Antoine Pitrou: > On Fri, 30 Aug 2013 00:10:27 +0200 Christian Heimes > wrote: >> -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 >> >> Hello, >> >> Coverity has published it

Re: [Python-Dev] Add function to signal module for getting main thread id

2013-08-30 Thread Christian Heimes
Am 30.08.2013 11:39, schrieb Antoine Pitrou: > > Le Fri, 30 Aug 2013 12:24:07 +0300, > Andrew Svetlov a écrit : >> Main thread is slightly different from others. >> Signals can be subscribed from main thread only. >> Tulip has special logic for main thread. >> In application code we can explicitl

Re: [Python-Dev] Coverity Scan Spotlight Python

2013-08-30 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am 30.08.2013 01:24, schrieb Sturla Molden: > > Do the numbers add up? > > .005 defects in 1,000 lines of code is one defect in every 200,000 > lines of code. > > However they also claim that "to date, the Coverity Scan service > has analyzed near

Re: [Python-Dev] peps: document requirements

2013-09-30 Thread Christian Heimes
Am 30.09.2013 20:16, schrieb Antoine Pitrou: > On Mon, 30 Sep 2013 17:08:00 +0200 (CEST) > christian.heimes wrote: >> + >> +* It should return ``0`` for zero length input. (Note: This can be handled >> as >> + special case, too.) > > What is this required for? The main requirement is that equal

[Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-03 Thread Christian Heimes
Hi, some of you may have seen that I'm working on a PEP for a new hash API and new algorithms for hashing of bytes and str. The PEP has three major aspects. It introduces DJB's SipHash as secure hash algorithm, chances the hash API to process blocks of data instead characters and it adds an API to

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-03 Thread Christian Heimes
Am 03.10.2013 21:05, schrieb Guido van Rossum: > Hm. I would like to stick to the philosophy that Python's hash > should be as fast as it possibly can be, and should not be mistaken > for a cryptographic hash. The point is to optimize dict lookups, > nothing more, given typical (or even atypical) k

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-03 Thread Christian Heimes
Am 03.10.2013 21:45, schrieb Guido van Rossum: > But fixing that shouldn't need all the extra stuff you're > proposing. I have proposed some of the extra stuff for more flexibility, the rest is for testing and debugging. > What's a Python randomization key? Python's hash randomization key, the s

Re: [Python-Dev] PEP 456

2013-10-03 Thread Christian Heimes
Am 03.10.2013 21:53, schrieb Serhiy Storchaka: >> the first time time with a bit shift of 7 > > Double "time". thx, fixed >> with a 128bit seed and 64-bit output > > Inconsistancy with hyphen. There are same issues in other places. I have unified the use of hyphens, thx! >> bytes_hash provide

Re: [Python-Dev] Make str/bytes hash algorithm pluggable?

2013-10-04 Thread Christian Heimes
Am 04.10.2013 11:15, schrieb Victor Stinner: > 2013/10/4 Armin Rigo : >> The current hash randomization is simply not preventing anything; >> someone posted long ago a way to recover bit-by-bit the hash >> randomized used by a remote web program in Python running on a >> server. > > Oh interesting

Re: [Python-Dev] Python startup time

2013-10-09 Thread Christian Heimes
Am 09.10.2013 14:39, schrieb Benjamin Peterson: > There's also the increasing number of modules (Python or otherwise) > that have to be loaded on startup. I have gathered some stats: Python 3.4: >>> len(sys.modules) 56 >>> len(list(name for name in sys.modules if name not in sys.buil

Re: [Python-Dev] Python startup time

2013-10-09 Thread Christian Heimes
Am 09.10.2013 14:54, schrieb Christian Heimes: > Am 09.10.2013 14:39, schrieb Benjamin Peterson: >> There's also the increasing number of modules (Python or otherwise) >> that have to be loaded on startup. We can easily peel off about 11 modules (re, sysconfig and their dep

Re: [Python-Dev] Python startup time

2013-10-10 Thread Christian Heimes
Am 10.10.2013 02:18, schrieb Eric Snow: > On Wed, Oct 9, 2013 at 8:30 AM, Christian Heimes > wrote: >> The os module imports MutableMapping from collections.abc. That >> import adds collections, collections.abc and eight more modules. >> I'm not sure if we can

Re: [Python-Dev] Python startup time

2013-10-10 Thread Christian Heimes
Am 10.10.2013 06:41, schrieb yoav glazner: > I'm not sure Droping imports is the best way to go, since every python > script/app will import common modules right on the start and it will > still seem like the interpeter boot is slow. > > making modules load faster seems like a better approch Not

Re: [Python-Dev] cpython: Issue #19209: Remove import of copyreg from the os module to speed up

2013-10-12 Thread Christian Heimes
Am 12.10.2013 01:32, schrieb Nick Coghlan: >> diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py >> --- a/Lib/test/test_site.py >> +++ b/Lib/test/test_site.py >> @@ -431,10 +431,13 @@ >> modules = eval(stdout.decode('utf-8')) >> self.assertIn('site', modules) >> >> +

Re: [Python-Dev] Change PEP 399 import recommendation

2013-10-12 Thread Christian Heimes
Am 12.10.2013 17:37, schrieb Nick Coghlan: > I think the default recommendation in PEP 399 still makes sense - 2 > modules are easy to manage than three and the idiom allows for easy > partial replacement. We could ues yet another approach and put the pure Python implementation of modules into ano

[Python-Dev] Right place for PBKDF2 wrapper

2013-10-12 Thread Christian Heimes
Hi, I have written a interface to OpenSSL's PKCS5_PBKDF2_HMAC() function. It implements PKCS#5's password based key derivation function 2 with HMAC as pseudo-random function. It supports any digest that is supported by OpenSSL, e.g. SHA-1, SHA-256 and SHA-512. It's a low level inteface that takes

Re: [Python-Dev] Right place for PBKDF2 wrapper

2013-10-12 Thread Christian Heimes
Am 12.10.2013 23:04, schrieb Gregory P. Smith: > agreed with any of these three. I'm going for hashlib.pbkdf2_hmac() for now. Right now it's just one function. We can always add a new module for a high level interface later. > that also makes sense... > > I'd leave hmac.py around through at leas

Re: [Python-Dev] Change PEP 399 import recommendation

2013-10-12 Thread Christian Heimes
Am 12.10.2013 19:02, schrieb Antoine Pitrou: > If you grow sys.path, imports get slower. > (IMHO it also makes the source tree more cumbersome to navigate) It shouldn't slow down successful imports. The module finder stops at first hit. A directory with pure Python modules is less cumbersome than

Re: [Python-Dev] Right place for PBKDF2 wrapper

2013-10-13 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Am 13.10.2013 08:32, schrieb Nick Coghlan: > +1 to hashlib from me (especially since we used that as the best > available home for compare_digest). I'm afraid your memory doesn't serve you well. :( compare_digest is implemented in _operator.c. Its

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Christian Heimes
On 2016-06-09 13:25, Larry Hastings wrote: > > A problem has surfaced just this week in 3.5.1. Obviously this is a > good time to fix it for 3.5.2. But there's a big argument over what is > "broken" and what is an appropriate "fix". > > As 3.5 Release Manager, I can put my foot down and make ru

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Christian Heimes
On 2016-06-09 19:14, Steven D'Aprano wrote: > On Thu, Jun 09, 2016 at 12:39:00PM -0400, Donald Stufft wrote: > >> There are three options for what do with os.urandom by default: >> >> * Allow it to silently return data that may or may not be >> cryptographically secure based on what the state of

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-09 Thread Christian Heimes
On 2016-06-10 05:48, Donald Stufft wrote: > >> On Jun 9, 2016, at 11:11 PM, Larry Hastings > > wrote: >> >> I don't understand why so many people seem to think it's okay to break >> old code in new versions of Python, when Python's history has shown a >> similarly strong

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-11 Thread Christian Heimes
On 2016-06-10 20:42, Chris Jerdonek wrote: > On Fri, Jun 10, 2016 at 11:29 AM, David Mertz wrote: >> This is fairly academic, since I do not anticipate needing to do this >> myself, but I have a specific question. I'll assume that Python 3.5.2 will >> go back to the 2.6-3.4 behavior in which os.u

Re: [Python-Dev] BDFL ruling request: should we block forever waiting for high-quality random bits?

2016-06-11 Thread Christian Heimes
On 2016-06-11 16:37, Victor Stinner wrote: >> I repeat for like the fifth time: > > So, is there a candidate to write a PEP? > > I didn't read the thread. As expected, the discussion restarted for the > 3rd time, there are almost 100 emails in this thread. Sorry, I'm out. I simply lack the nec

Re: [Python-Dev] New hash algorithms: SHA3, SHAKE, BLAKE2, truncated SHA512

2016-06-12 Thread Christian Heimes
On 2016-05-25 12:29, Christian Heimes wrote: > Hi everybody, > > I have three hashing-related patches for Python 3.6 that are waiting for > review. Altogether the three patches add ten new hash algorithms to the > hashlib module: SHA3 (224, 256, 384, 512), SHAKE (SHA3 XOF 128,

Re: [Python-Dev] frame evaluation API PEP

2016-06-20 Thread Christian Heimes
On 2016-06-20 22:18, Guido van Rossum wrote: > On Mon, Jun 20, 2016 at 12:59 PM, Brett Cannon > wrote: > > MRAB's response made me think of a possible approach: the > co_extra field could be the very last field of the PyCodeObject > struct and only

Re: [Python-Dev] Should we fix these errors?

2016-07-22 Thread Christian Heimes
From f84cfa464e4b7d03776afabe9c0819d491c5617b Mon Sep 17 00:00:00 2001 From: Christian Heimes Date: Fri, 19 Feb 2016 16:22:23 +0100 Subject: [PATCH 04/13] Fix dereferencing before NULL check in _PyState_AddModule() _PyState_AddModule() accesses a member of PyModuleDef* def first and then check def

Re: [Python-Dev] Should we fix these errors?

2016-07-23 Thread Christian Heimes
On 2016-07-22 16:36, Guido van Rossum wrote: > Somebody did some research and found some bugs in CPython (IIUC). The > published some questionable fragments. If there's a volunteer we could > probably easily fix these. (I know we already have occasional Coverity > scans and there are other tools to

[Python-Dev] socket.setsockopt() with optval=NULL

2016-08-21 Thread Christian Heimes
Hi, the socket.setsockopt(level, optname, value) method has two calling variants. When it is called with a buffer-like object as value, it calls the C API function setsockopt() with optval=buffer.buf and optlen=buffer.len. When value is an integer, setsockopt() packs it as int32 and sends it with

[Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Christian Heimes
Hi, we need to talk about OpenSSL and LibreSSL before the next release of Python. I'm working on a PEP. Most likely it won't be ready before the feature freeze. But first let me start with some good news. OpenSSL 1.1 was released a couple of days ago. It changed a lot of aspects of its internal A

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-28 Thread Christian Heimes
On 2016-08-29 04:38, Ned Deily wrote: > On Aug 28, 2016, at 19:06, Benjamin Peterson wrote: >> On Sun, Aug 28, 2016, at 13:40, Christian Heimes wrote: >>> Here is the deal for 2.7 to 3.5: >>> >>> 1) All versions older than 0.9.8 are completely out-of-scope and

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-29 Thread Christian Heimes
On 2016-08-29 10:09, M.-A. Lemburg wrote: > On 28.08.2016 22:40, Christian Heimes wrote: >> ... >> I like to reduce the maintenance burden and list of supported OpenSSL >> versions ASAP. OpenSSL has deprecated 0.9.8 and 1.0.0 last year. 1.0.1 >> will reach EOL by the

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-29 Thread Christian Heimes
On 2016-08-29 21:31, M.-A. Lemburg wrote: > On 29.08.2016 18:33, Cory Benfield wrote: >> >>> On 29 Aug 2016, at 04:09, M.-A. Lemburg wrote: >>> >>> On 28.08.2016 22:40, Christian Heimes wrote: >>>> ... >>>> I like to reduce the mainten

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-29 Thread Christian Heimes
On 2016-08-29 22:10, Random832 wrote: > On Mon, Aug 29, 2016, at 04:09, M.-A. Lemburg wrote: >> Hmm, that last part would mean that Python 3.7 will no longer compile >> on e.g. Ubuntu 14.04 LTS which uses OpenSSL 1.0.1 as default version. >> Since 14.04 LTS is supported until 2019, I think it would

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-29 Thread Christian Heimes
On 2016-08-29 23:00, Gregory P. Smith wrote: > > Given that you already said: > > """ > For 3.6 I don't require any 1.0.2 feature yet. The 1.1.0 patch keeps > code compatible with 0.9.8zc to 1.1.0. But as soon as I use new > features, the ssl module will no longer be source and build compatible >

Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects

2016-08-29 Thread Christian Heimes
On 2016-08-29 23:38, Brett Cannon wrote: > That means we still want to find a solution to attach arbitrary data to > code objects without sacrificing performance. One proposal is what's in > PEP 523 for the extra field. Another option is to make the memory > allocator for code objects pluggable and

Re: [Python-Dev] Update on PEP 523 and adding a co_extra field to code objects

2016-08-30 Thread Christian Heimes
On 2016-08-30 01:14, Brett Cannon wrote: > So the struct in question can be found at > https://github.com/python/cpython/blob/2d264235f6e066611b412f7c2e1603866e0f7f1b/Include/code.h#L10 > . > The official docs say the fields can be changed at any time, so > re-arranging them shouldn't break any AB

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-31 Thread Christian Heimes
On 2016-08-30 22:07, M.-A. Lemburg wrote: > That was not my point. It's unfortunate that Python depends on > a library which is inevitably going to need updates frequently, > and which then may have the implication that Python won't compile on > systems which don't ship with more recent OpenSSL lib

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-31 Thread Christian Heimes
On 2016-08-31 10:31, M.-A. Lemburg wrote: > In all this discussion I have yet to find a compelling security > relevant argument for using an 1.0.2 API which is so important > that we cannot make this optional at runtime. > > The only argument Christian reported was this one: > > """ >> BTW: Are t

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-31 Thread Christian Heimes
On 2016-08-31 11:33, M.-A. Lemburg wrote: > On 31.08.2016 10:50, Christian Heimes wrote: >> On 2016-08-31 10:31, M.-A. Lemburg wrote: >>> In all this discussion I have yet to find a compelling security >>> relevant argument for using an 1.0.2 API which is so important &

Re: [Python-Dev] Supported versions of OpenSSL

2016-08-31 Thread Christian Heimes
On 2016-08-30 18:00, Antoine Pitrou wrote: > On Sun, 28 Aug 2016 22:40:11 +0200 > Christian Heimes wrote: >> >> Here is the deal for 2.7 to 3.5: >> >> 1) All versions older than 0.9.8 are completely out-of-scope and no >> longer supported. >> >> 2)

[Python-Dev] Patch reviews

2016-08-31 Thread Christian Heimes
Hi, I have 7 patches for 3.6 ready for merging. The new features were discussed on Security-SIG and reviewed by Victor or GPS. The patches just need one final review and an ACK. The first three patches should land in 2.7, 3.4 and 3.5, too. http://bugs.python.org/issue26470 Make OpenSSL module com

Re: [Python-Dev] Patch reviews

2016-09-01 Thread Christian Heimes
On 2016-08-31 22:31, Christian Heimes wrote: > Hi, > > I have 7 patches for 3.6 ready for merging. The new features were > discussed on Security-SIG and reviewed by Victor or GPS. The patches > just need one final review and an ACK. The first three patches should > land in 2.7

Re: [Python-Dev] Tweak to PEP 523 for storing a tuple in co_extra

2016-09-03 Thread Christian Heimes
On 2016-09-04 00:03, Yury Selivanov wrote: > > > On 2016-09-03 12:27 PM, Brett Cannon wrote: >> Below is the `co_extra` section of PEP 523 with the update saying that >> users are expected to put a tuple in the field for easier simultaneous >> use of the field. >> >> Since the `co_extra` discussi

Re: [Python-Dev] Patch reviews

2016-09-04 Thread Christian Heimes
On 2016-09-01 23:15, Victor Stinner wrote: > 2016-08-31 22:31 GMT+02:00 Christian Heimes : >> https://bugs.python.org/issue27744 >> Add AF_ALG (Linux Kernel crypto) to socket module > > This patch adds a new socket.sendmsg_afalg() method on Linux. > > "afal

[Python-Dev] TLS handshake performance boost

2016-09-05 Thread Christian Heimes
Hi, I have yet another patch for the ssl module, http://bugs.python.org/issue19500 . The patch adds support for SSL session resumption on the client side. A SSLContext automatically handles server-side session. SSL sessions speed up successive TLS connections to the same host considerable. My naïv

[Python-Dev] hg push segfault

2016-09-08 Thread Christian Heimes
Hi, About 10 minutes ago I got a couple of remote segfaults from hg.python.org. They occurred during push and pull operations: $ hg push pushing to ssh://h...@hg.python.org/cpython remote: bash: line 1: 25019 Segmentation fault HGPUSHER=christian.heimes /srv/hg/bin/hg-ssh /srv/hg/repos/* abort: n

Re: [Python-Dev] cpython (3.5): supress coroutine warning when an exception is pending (#27968)

2016-09-08 Thread Christian Heimes
On 2016-09-07 17:47, benjamin.peterson wrote: > https://hg.python.org/cpython/rev/234f758449f8 > changeset: 103223:234f758449f8 > branch: 3.5 > parent: 103213:7537ca1c2aaf > user:Benjamin Peterson > date:Wed Sep 07 08:46:59 2016 -0700 > summary: > supress coroutine wa

[Python-Dev] Let's make the SSL module sane

2016-09-10 Thread Christian Heimes
Hi, (CC TLS gurus) For 3.6 I like to make the SSL more sane and more secure by default. Yes, I'm a bit late but all my proposals are implemented, documented, partly tested and existing tests are passing. I'm going to write more tests and documentation after beta1. First I like to deprecated so

Re: [Python-Dev] Let's make the SSL module sane

2016-09-10 Thread Christian Heimes
On 2016-09-10 18:24, Donald Stufft wrote: > >> On Sep 10, 2016, at 10:22 AM, Christian Heimes wrote: >> >> I don't load any certs because it is not possible to remove a cert or >> X509 lookup once it is loaded. create_default_context() just have to >> lo

Re: [Python-Dev] Let's make the SSL module sane

2016-09-10 Thread Christian Heimes
On 2016-09-10 17:24, Nick Coghlan wrote: > On 11 September 2016 at 00:22, Christian Heimes wrote: >> First I like to deprecated some old APIs and favor of SSLCotext. We have >> multiple ways to create a SSL socket or to configure libraries like >> urllib. The general idea

Re: [Python-Dev] cpython: Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal.

2016-09-12 Thread Christian Heimes
On 2016-09-09 18:53, guido.van.rossum wrote: > https://hg.python.org/cpython/rev/804b71d43c85 > changeset: 103415:804b71d43c85 > user:Guido van Rossum > date:Fri Sep 09 09:36:26 2016 -0700 > summary: > Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal.

Re: [Python-Dev] cpython: Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal.

2016-09-12 Thread Christian Heimes
On 2016-09-12 11:46, Ivan Levkivskyi wrote: > Christian, > > When I wrote this, my intention was like: cur & DEF_LOCAL is a "more > serious" error, so that if both errors are made in the same statement: > def f(): > x: int = 5 > global x > > "SyntaxError: global after assignment" will be

Re: [Python-Dev] cpython: Issue #27999: Make "global after use" a SyntaxError, and ditto for nonlocal.

2016-09-12 Thread Christian Heimes
On 2016-09-12 12:29, Ivan Levkivskyi wrote: > On 12 September 2016 at 12:24, Christian Heimes <mailto:christ...@python.org>> wrote: > > The code looks suspicious. Can you please > provide a patch that makes it more obvious, e.g. either by using if / > els

Re: [Python-Dev] cpython (3.6): replace usage of Py_VA_COPY with the (C99) standard va_copy

2016-09-21 Thread Christian Heimes
On 2016-09-21 05:39, benjamin.peterson wrote: > https://hg.python.org/cpython/rev/278b21d8e86e > changeset: 103977:278b21d8e86e > branch: 3.6 > parent: 103975:d31b4de433b7 > user:Benjamin Peterson > date:Tue Sep 20 20:39:33 2016 -0700 > summary: > replace usage of Py_

[Python-Dev] Code quality report

2016-09-24 Thread Christian Heimes
Hi, here is a short code quality report. Overall we are in a good shape for Python 3.6.0. I'm a bit worried about the amount of security bugs, though. Some haven't progressed in more than a year. Coverity Scan - 3.6.0b1 added a bunch of new defects, most of them were false positives

Re: [Python-Dev] Tests breakage with latest openssl (issue28689)

2016-11-28 Thread Christian Heimes
On 2016-11-28 16:35, Charalampos Stratakis wrote: > Hello, > > Escalating here an issue that seems that it should be tagged as blocker. > > Currently the latest version of openssl-1.1.0c breaks Python's test suite. > > The problematic commit was identified [0] and reverted [1] at openssl's > up

Re: [Python-Dev] Tests breakage with latest openssl (issue28689)

2016-11-29 Thread Christian Heimes
On 2016-11-28 18:16, Charalampos Stratakis wrote: > Hi Christian and thanks for the fast reply, > > It's great to hear that the latest version is working fine. Do you have > anymore details on the fix/breakage? > > The latest commit at Fedora's rawhide openssl package is at [0]. Is it > missin

[Python-Dev] 3.6.0: OpenSSL 1.1.0c is not supported

2016-12-14 Thread Christian Heimes
Hi Ned, please add a reminder to the release docs that Python 3.6.0 is not compatible with OpenSSL 1.1.0c, https://bugs.python.org/issue28689. 1.1.0 to 1.1.0b work fine. 1.1.0d will be compatible, too. Regards, Christian signature.asc Description: OpenPGP digital signature

Re: [Python-Dev] Deprecate `from __future__ import unicode_literals`?

2016-12-17 Thread Christian Heimes
On 2016-12-17 10:06, Serhiy Storchaka wrote: > On 16.12.16 21:24, Guido van Rossum wrote: >> e.g. the argument to getattr() -- I still hear of code that breaks due >> to this occasionally) > > What is the problem with unicode in getattr()? Unicode attribute name is > converted to str, and since th

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

2017-01-20 Thread Christian Heimes
On 2017-01-20 13:15, INADA Naoki wrote: >> >> "this script counts static memory usage. It doesn’t care about dynamic >> memory usage of processing real request" >> >> You may be trying to optimize something which is only a very small >> fraction of your actual memory footprint. That said, the mars

Re: [Python-Dev] Investigating Python memory footprint of one real Web application

2017-01-20 Thread Christian Heimes
On 2017-01-20 13:15, INADA Naoki wrote: >> >> "this script counts static memory usage. It doesn’t care about dynamic >> memory usage of processing real request" >> >> You may be trying to optimize something which is only a very small >> fraction of your actual memory footprint. That said, the mars

Re: [Python-Dev] SSL certificates recommendations for downstream python packagers

2017-01-30 Thread Christian Heimes
On 2017-01-30 14:53, David Cournapeau wrote: > Hi, > > I am managing the team responsible for providing python packaging at > Enthought, and I would like to make sure we are providing a good (and > secure) out of the box experience for SSL. > > My understanding is that PEP 476 is the latest PEP t

Re: [Python-Dev] SSL certificates recommendations for downstream python packagers

2017-01-30 Thread Christian Heimes
On 2017-01-30 21:50, Cory Benfield wrote: > > >> On 30 Jan 2017, at 13:53, David Cournapeau wrote: >> >> Are there any official recommendations for downstream packagers beyond PEP >> 476 ? Is it "acceptable" for downstream packagers to patch python's default >> cert locations ? > > There *are

Re: [Python-Dev] SSL certificates recommendations for downstream python packagers

2017-01-30 Thread Christian Heimes
On 2017-01-30 22:00, David Cournapeau wrote: > > > On Mon, Jan 30, 2017 at 8:50 PM, Cory Benfield > wrote: > > > > > On 30 Jan 2017, at 13:53, David Cournapeau > wrote: > > > > Are there any official recommendations for downstr

Re: [Python-Dev] SSL certificates recommendations for downstream python packagers

2017-01-30 Thread Christian Heimes
On 2017-01-30 22:19, David Cournapeau wrote: > Hm. Is this documented anywhere ? We have customers needing > "private/custom" certificates, and I am unsure where to look for. For full control it is advised to use a custom SSLContext that only loads the internal CA. https://docs.python.org/3/librar

<    1   2   3   4   5   6   7   8   9   10   >