[Python-Dev] Re: Security releases of CPython
On Fri, 2021-02-19 at 17:03 -0500, Terry Reedy wrote: > On 2/19/2021 5:11 AM, Michał Górny wrote: > > On Thu, 2021-02-11 at 23:24 -0500, Terry Reedy wrote: > > > > Releases are not just a push of a button. Make the release > > > job too onerous, and there might not be any more volunteers. > > > > While I understand your concerns and sympathize with them, > > Your accusations of unfairness in response to my polite, volunteered > response, suggest otherwise. > > > I don't think it's fair to play the volunteer card here. > ... > > it just feels unfair for you to be dumping the security work on us. > > Which I did not do. Bye. I'm sorry, I did not mean to insult you. That's just how I felt after reading the response. -- Best regards, Michał Górny ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GJVNYYMEE73GGYELTRR66XUF2EPAJNYF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Security releases of CPython
On Thu, Feb 11, 2021 at 9:44 PM Michał Górny wrote: > I feel that vulnerability fixes do not make it to end users fast enough. I think that it's time to put that into perspective with past vulnerabilities. Ok, let me look at the timeline of the discussed vulnerability, ctypes CVE-2021-3177: https://python-security.readthedocs.io/vuln/ctypes-buffer-overflow-pycarg_repr.html 2021-01-16: Python issue bpo-42938 reported by Jordy Zomer ... 2021-01-18 (+2 days): commit c347cbe (branch 3.9) 2021-01-18 (+2 days): commit ece5dfd (branch 3.8) 2021-01-19 (+3 days): CVE-2021-3177 published ... 2021-02-19 (+34 days): Python 3.8.8 released 2021-02-19 (+34 days): Python 3.9.2 released Ok. What about vulnerabilities fixes released last years? "HTTP header injection in urllib, urrlib2, httplib and http.client modules" https://python-security.readthedocs.io/vuln/http-header-injection.html 2017-09-19 (+1030 days): Python 3.3.7 released "CGI directory traversal" https://python-security.readthedocs.io/vuln/cgi-directory-traversal-is_cgi.html 2011-05-09 (+1158 days): CVE-2011-1015 published 2013-04-07 (+1857 days): Python 3.2.4 released 2013-04-07 (+1857 days): Python 3.3.1 released "httplib unlimited read" https://python-security.readthedocs.io/vuln/httplib-unlimited-read.html 2011-06-11 (+652 days): Python 2.7.2 released 2011-06-11 (+652 days): Python 3.1.4 released "rgbimg and imageop overflows" https://python-security.readthedocs.io/vuln/rgbimg-imageop-overflows.html 2008-12-19 (+460 days): Python 2.5.3 released So the CVE-2021-3177 fix was delivery between 14x and 55x faster than the other listed fixes (I picked a few worst cases to put numbers in perspective). Congrats to the core developers for fixing the vulnerability in only *3* days and to release manager for releasing *4* (!) Python versions (3.6.13, 3.7.10, 3.8.8, 3.9.2) in only 34 days! I would like to highlight that exploiting a directory traversal or HTTP header injection is really trivial. Once you find a pattern to explore the filesystem / inject a HTTP header, the exploit is 100% reliable. On the other side, there is no known exploit for ctypes CVE-2021-3177 and ctypes is rarely used. I read that Django's GIS uses ctypes and floats, but so far nobody shows that PyCArg_repr() is called, and nobody published an exploit. To write a CVE-2021-3177 exploit, you must create a 64-bit floating point number (only 8 bytes!) which becomes valid machine code, and this code should allow to take control on the machine, once it's formatted as decimal. For example, PyCArg_repr(123.5) writes "123.5\0" string into the stack memory. but I don't think that it's valid x86-64 machine code. It is also hard to write a reliable exploit by injecting machine code in the stack memory. --- Nowadays it's way more difficult than 10 years ago to write an exploit using a stack overflow, C compilers provide multiple hardening layers: - FORTIFY_SOURCE, - Control flow Enforcement Technology (Intel CET), - Address Space Layout Randomization (ASLR), - stack protector, - Position Independent Executable (PIE), - etc. See https://wiki.debian.org/Hardening for example of C flags and linker flags for harderning. Did anyone notice that Red Hat Entreprise Linux 8 (RHEL) is *not* affected by the ctypes CVE-2021-3177 vulnerability thanks to hardening? "Red Hat Enterprise Linux 8: python36:3.6/python36: Not affected" and "This flaw could have had a higher Impact, however our packages are compiled with FORTIFY_SOURCE, which provides runtime protection to some memory and string functions and prevents this flaw from actually overwriting the buffer and potentially executing code." => https://access.redhat.com/security/cve/cve-2021-3177 I suggest you checking how your operating system built your Python executable, and libpython if Python is built in shared mode: hardening can prevent the ctypes vulnerabiliy, but also against *future* vulnerabilities! For example, Fedora 33 builds Python 3.9 with the following C flags: -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fcf-protection -fstack-clash-protection -fstack-protector-strong -fPIC -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 => add the -fPIE flag (...) And linker flags: -Wl,-z,now -Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld => add the -pie flag (...) You can inspect hardening options on a ELF binary using hardening-check tool: $ hardening-check /usr/bin/python3.9 /usr/bin/python3.9: Position Independent Executable: yes Stack protected: no, not found! Fortify Source functions: unknown, no protectable libc functions used Read-only relocations: yes Immediate binding: yes Stack clash protection: unknown, no -fstack-clash-protection instructions found Control flow integrity: yes $ hardening-check /usr/lib64/libpython3.9.so.1.0 /usr/lib64/libpython3.9.so.1.0: Position Independent Executable: no, regular shared library (ignored) Stack protected: yes Fortify Source functions: yes (
[Python-Dev] Re: PEP 653: Precise Semantics for Pattern Matching
Hi Oscar Quoting Oscar Benjamin : On Fri, 19 Feb 2021 at 15:41, Tobias Kohn wrote: [...] It's not easy now to look back over the history of all of this. My recollection of the original version of PEP 622 (if that's the right one...) was that it had an overcomplicated protocol for __match__. It needed to be simplified but in the end it was dropped. Without further context, I would agree with you: it is difficult to look back over the entire history of the pattern matching PEPs. However, on the one hand PEP 622 is directly linked in the current PEP 634. On the other hand, for those who actively particiated in the discussion of both PEP 622 as well as the DLS paper, I find it a bit too easy and 'convenient' to call those resources now 'hard to find'. That being said, I think it is part of your homework to first research about the history and what others have done before proposing your innovation. Granted, this is hard and very laborious work that often takes a long time and can be frustrating. But if we want to make progress and move forward, we have to stop running in circles.—To be absolutely clear here: I do not think that Mark's proposal is running in circles and I think it is fair enough to bring up these ideas. But I equally think it is well possible to acknowledge that one part of it is a discussion of existing ideas, and to have a look at why these ideas have not made it into PEP 634. The question now is if it will be straight-forward to retrofit a protocol to PEP 634 after it is released and when backward compatibility constraints kick in. PEP 653 (as discussed here) is precisely an attempt to retrofit a protocol to PEP 634. I think the difficulties involved in achieving that will become harder rather than easier in future. -- Oscar There are actually two protocols in PEP 653. One is about changing the fundamental design of pattern matching as outlined in PEP 634. This is a part that I reject for reasons presented in one of my last posts. The second one is the customisation part using `__deconstruct__` or `__match__`. This is something that I like a lot and would certainly like to see in pattern matching—alas, just being in favour of something is not a strong enough argument for it. Similar to my votum above: if we are going to discuss this, I think we need new arguments or evidence, something to show why the previous decision to drop it (for now) was wrong. For that it might be worth reading the respective section in deferred ideas of PEP 622. Some part of our decision was based on the notion that adding such a custom protocol later one will be relatively painless, but if we were wrong, please speak up and show us what we have overlooked. If you can present, for instance, some real-world use cases where this feature would enormously benefit sympy (or any other popular package), that would be helpful and a good starting point. Kind regards, Tobias ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7EK4K36F6NKD67VTBS2XWRXAN4E737AD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Security releases of CPython
On 2/19/21 11:55 PM, Steve Holden wrote: The PSF needs needs sufficient money to hire a couple of people, so the PSF can turn release management and security maintenance from unpaid volunteers into paid fulltime jobs. Oh, is that all? Sustainability of the PSF, as has been shown over its 20 years of existence, is not an easy thing to achieve. It was hit by the financial crisis in 2008 and again by the coronavirus crisis last year, both things that affected all foundations. If you plan to bring this kind of money in and rely on it, you have to ensure it comes from sources that can't just be switched off when budgets tighten. It could be done, but "easy" sounds like exaggeration to me. Unless your suggestions were joking, but I saw no smiley ... Steve's comments probably need no reinforcement, but I can speak as someone who's been funded by fees collected from motivated companies, and had the tap turn off. In my cases (more than one) they were consortia where the members committed a set fee yearly, and then one year: we've decided not to renew, from one or more... It's actually easier to raise funds for a one-time campaign than one that is committed to last for several years, in my limited experience on the fund-raising side. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VMPYGK6MDPFUCNSNB62BFCVEEDUQW4D6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Anyone interested in leading a sprint at PyCascades tomorrow?
It's short notice, but PyCascades is hosting sprints on Sunday, Feb. 21 from 09:00 - 12:30 PST tomorrow and the organizers wanted to see if anyone happened to be up for leading or helping out anyone who wants to sprint on CPython. https://2021.pycascades.com/program/sprints/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VWI3JUSECIX4HT6YAMRAXG54V3OBGS4I/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Security releases of CPython
Looking at the other replies, I'm wondering if you fully understand python's variant of version numbering. I suggest we change the announcement template from: "Python 3.9.2 is the newest major release of the Python programming language, and it contains many new features and optimizations." to: "Python 3.9 was the newest major release of the Python programming language, and it contains many new features and optimizations. Python 3.9.2 is a bugfix and security release; it has no new features relative to 3.9." 3.9.1 (and 3.9.27, if that ever happens) are supposed to the be the same as 3.9.0, except with bugs fixed. Because "a feature is just a bug with tenure", there comes a time when non-security bugs stop being fixed. There isn't a hard-and-fast rule on when that is, but within a year or two. Looking back, a .6 release was unlikely to contain much beyond security. Even before that time, CPython still tries to err on the overly-cautious side of "can this bug-fix be backported", because of an overly-optimistic assessment ~20 years ago. (A harmless feature was added in a backwards-compatible way, but for a while instructions had to specify a bugfix version as well.) In theory, someone could release 3.9.0s1, 3.9.0s2, ... 3.9.1s1 ... but what would be the point? 3.9.1s2 would have contained exactly the same changes as 3.9.2rc, which apparently didn't get picked up much. The difference between 3.9.2rc and 3.9.2 does include a non-security bugfix -- a part of the Windows API that was advertised as working will now actually work. Is that really adding much extra upgrade risk? Of course, to get these extra releases, someone will have to be more careful about deciding what counts as a security fix vs a regular bugfix, which is already sometimes fuzzy. And realistically, it is *only* the security fixes that are likely to be a problem for working code ... Even if the releases were trivial, how much value would that actually provide? -jJ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WDBRC4WSOYSHMQVYDWHWSO2ITH4YGRQC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Security releases of CPython
On Thu, 2021-02-11 at 23:24 -0500, Terry Reedy wrote: > ... Releases are not just a push of a button. On 2021-02-19 15:05, Stestagg wrote: > > The thing that stood out from this conversation, for me, is: Releases > > are too hard, and there’s a risk of not having enough volunteers as a > > result. > > > > How hard is it to fix that? > > Are there no technical solutions that might help reduce the cost? Sounds like automating until it is "just a push of a button," should be a goal. According to Victor there has been progress, but always room for more. On 2021-02-19 18:36, Brett Cannon wrote: There is no specific drive to hire someone to target security and/or release management at the moment. We just got enough funding for the first time to hire a dev-in-residence for Python itself to try to help tackle our 1.4K PR backlog in some fashion that they won't be dedicated to security or release management. Looking at the PSF Annual report from a normal year it also looks like there is substantial revenue, income, and assets available. There are significant expenses as well. (And of course income down for ~two years, but it should return at some point.) I would argue that security releases are of higher importance than most, including sponsorship programs and grants, and that a mild change of priorities is in order. This is not to say (of course) that any other categories are not important, simply that having machines and networks pwned while being sponsored or educated is a devil's bargain. Such a position may not require a hundred-thousand a year salary either. A skilled contractor could improve automation, while a stipend might be enough to ensure releases get out within a ~week if they need to. -Mike ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MWDLKT3H4CXVPGSSMYSXTRBI33637LWA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Deprecate support for mingw - add to PEP 11
I think perhaps we should admit that this build system is no longer supported. From everything I can tell, the mingw project is no longer maintained. The project's site, mingw.org, is no longer live; the project on sourceforge, although still downloaded daily, had its last commit almost 3 years ago - a commit which changed the official project URI to a new link that now is also dead. Looking over BPO there are a little over 50 bugs open against mingw, but only 7 that have any meaningful activity within the last three years. Three of those issues explicitly mention mingw-w64 which is an active fork of the original mingw project (active homepage, commits almost daily, new release within the last 6 months) and I wonder if this is the project the other 4 projects meant by "mingw"? Ideally any features and flags in the code base for mingw would be checked to already be working with mingw-w64 or else modified to work, but this would require a sponsor for this platform, which appears to be missing. Further, there is no buildbot for mingw, which should be a requirement to be a fully supported platform, as per this PEP. This potential work appears non-trivial with a cursory look at the mingw-w64-python pacman project, which contains close to 100 patch files. I am purposing instead that mingw be deprecated and, if a sponsor comes along, mingw-w64 can become re-supported, or newly supported depending on you point of view, as allowed by the PEP. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XIWF3OYL7OQRBVRBBQCBKPPJH5OKVVRC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Deprecate support for mingw - add to PEP 11
mingw-w64 might be a small change. But while one is it at, it might make sense to evaluate: https://clang.llvm.org/docs/MSVCCompatibility.html Apparently clang on Windows is working on calling convention compatibility with Visual Studio. On Sat, Feb 20, 2021 at 8:37 PM wrote: > I think perhaps we should admit that this build system is no longer > supported. From everything I can tell, the mingw project is no longer > maintained. The project's site, mingw.org, is no longer live; the project > on sourceforge, although still downloaded daily, had its last commit almost > 3 years ago - a commit which changed the official project URI to a new link > that now is also dead. > Looking over BPO there are a little over 50 bugs open against mingw, but > only 7 that have any meaningful activity within the last three years. Three > of those issues explicitly mention mingw-w64 which is an active fork of the > original mingw project (active homepage, commits almost daily, new release > within the last 6 months) and I wonder if this is the project the other 4 > projects meant by "mingw"? > Ideally any features and flags in the code base for mingw would be checked > to already be working with mingw-w64 or else modified to work, but this > would require a sponsor for this platform, which appears to be missing. > Further, there is no buildbot for mingw, which should be a requirement to > be a fully supported platform, as per this PEP. This potential work appears > non-trivial with a cursory look at the mingw-w64-python pacman project, > which contains close to 100 patch files. I am purposing instead that mingw > be deprecated and, if a sponsor comes along, mingw-w64 can become > re-supported, or newly supported depending on you point of view, as allowed > by the PEP. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/XIWF3OYL7OQRBVRBBQCBKPPJH5OKVVRC/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7K6YOTYPFPB6IJUVCJ5WH2DA65DDGH6T/ Code of Conduct: http://python.org/psf/codeofconduct/
