Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Thu, 21 Feb 2013 02:29:08 -0500 Tres Seaver wrote: > > Antoine, > > A single, small,, malicious XML file can kill a machine (not just the > process parsing it) by sucking all available RAM. We are talking hard > lockup, reboot-to-fix-it sorts of DOC here. Sure, but in many instances, reboot

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/21/2013 01:53 AM, Antoine Pitrou wrote: > On Thu, 21 Feb 2013 11:37:47 +1100 Steven D'Aprano > wrote: >> >> It's easy to forget that malware existed long before the Internet. >> The internet is just a transmission vector, it is not the source o

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Thu, 21 Feb 2013 10:38:07 +1000 Nick Coghlan wrote: > On Thu, Feb 21, 2013 at 9:49 AM, Tres Seaver wrote: > > Two words: "hash randomization". If it applies to one, it applies to > > the other. > > Agreed. Christian's suggested approach sounds sane to me: > > - make it possible to enable s

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Thu, 21 Feb 2013 11:37:47 +1100 Steven D'Aprano wrote: > > It's easy to forget that malware existed long before the Internet. The > internet is just a transmission vector, it is not the source of malicious > files. The source of malicious files is *other people*, and unless you never > use

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Wed, 20 Feb 2013 18:45:10 -0500 Donald Stufft wrote: > > No software you run on your computer grabs data from someone you don't trust > and it all validates that even though you trust them they haven't been > exploited? What the hell do you mean exactly? There are other reasons to validate d

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Stefan Behnel
Maciej Fijalkowski, 20.02.2013 21:17: > On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes wrote: >> Am 20.02.2013 17:25, schrieb Benjamin Peterson: >>> Are these going to become patches for Python, too? >> >> I'm working on it. The patches need to be discussed as they break >> backward compatibilit

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Feb 20, 2013, at 11:35 PM, Tres Seaver wrote: >I believe that the same rationale should apply as that for adding hash >randomization in 2.6.8: this is at least as bad a vulnerability, with >many more vectors of attack. Except that I really want

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2013 09:08 PM, Barry Warsaw wrote: > On Feb 21, 2013, at 10:38 AM, Nick Coghlan wrote: > >> - make it possible to enable safer behaviour globally in at least >> 2.7 and 3.3 (and perhaps in 2.6 and 3.2 security releases as well) > > I want to

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Barry Warsaw
On Feb 21, 2013, at 10:38 AM, Nick Coghlan wrote: >- make it possible to enable safer behaviour globally in at least 2.7 >and 3.3 (and perhaps in 2.6 and 3.2 security releases as well) I want to be fairly conservative with 2.6.9. -Barry ___ Python-Dev

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Fred Drake
On Wed, Feb 20, 2013 at 7:38 PM, Nick Coghlan wrote: > Christian's suggested approach sounds sane to me: Definitely. A strong +1 from me, FWIW these days. -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein ___ Py

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Vinay Sajip
Paul Moore gmail.com> writes: > Understood - that's why I suggested that distlib reach a point where > it's stable as an external package and supported on (some) older > versions. I'm hoping for an experience more like unittest2 than > packaging/distutils2. Currently, distlib runs on Python 2.6,

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Vinay Sajip
Chris Jerdonek gmail.com> writes: > Maybe this is already stated somewhere, but is there a plan for when > distlib will be brought into the repository? Is there a reason not to > do it now? It seems it would have more visibility that way (e.g. > people could see it as part of the development ve

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Vinay Sajip
M.-A. Lemburg egenix.com> writes: > The suggestion to have the metadata available on PyPI doesn't > have anything to do with security. > > It's about being able to determine compatibility and select the > right distribution file for download. The metadata also helps in > creating dependency grap

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Nick Coghlan
On Thu, Feb 21, 2013 at 9:49 AM, Tres Seaver wrote: > Two words: "hash randomization". If it applies to one, it applies to > the other. Agreed. Christian's suggested approach sounds sane to me: - make it possible to enable safer behaviour globally in at least 2.7 and 3.3 (and perhaps in 2.6 an

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Steven D'Aprano
On 21/02/13 10:22, Antoine Pitrou wrote: On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft wrote: On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A single 1 kB XML document can kill virtually any machine, eve

[Python-Dev] Postponing acceptance of PEP 426

2013-02-20 Thread Nick Coghlan
The feedback I have received (both on-list and in response to some off-list queries to specific people) tells me that PEP 426 isn't quite ready for acceptance yet. Things I'll be working on or facilitating over the next few weeks: - documenting an overall transition plan to put the new metadata f

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Donald Stufft
On Wednesday, February 20, 2013 at 6:22 PM, Antoine Pitrou wrote: > On Wed, 20 Feb 2013 18:21:22 -0500 > Donald Stufft mailto:donald.stu...@gmail.com)> > wrote: > > On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: > > > > It's not a distributed DoS issue, it's a severe DoS vulnera

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2013 06:22 PM, Antoine Pitrou wrote: > On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft > wrote: >> On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilitie

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Jesse Noller
On Feb 20, 2013, at 6:22 PM, Antoine Pitrou wrote: > On Wed, 20 Feb 2013 18:21:22 -0500 > Donald Stufft wrote: >> On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A single 1 kB XML document can kill

Re: [Python-Dev] [Distutils] PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread PJ Eby
On Tue, Feb 19, 2013 at 6:42 AM, Nick Coghlan wrote: > Nothing in the PEP is particularly original - almost all of it is > either stolen from other build and packaging systems, or is designed > to provide a *discoverable* alternative to existing > setuptools/distribute/pip practices (specifically,

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Donald Stufft
On Wednesday, February 20, 2013 at 6:23 PM, Christian Heimes wrote: > We can add a function to the XML package tree that enables all restrictions: > > * limit expansion depths of nested entities > * limit total amount of expanded chars > * disable external entity expansion > * optionally force exp

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread PJ Eby
On Wed, Feb 20, 2013 at 5:30 AM, M.-A. Lemburg wrote: > The wording in the PEP alienates the egg format by defining > an incompatible new standard for the location of the metadata > file: This isn't a problem, because there's not really a use case at the moment for eggs to include a PEP 426-forma

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft wrote: > On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: > > > It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A > > > single 1 kB XML document can kill virtually any machine, even servers > > > with more than

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 21.02.2013 00:08, schrieb Antoine Pitrou: > Not everyone is a security nuts. But, but, but ... it's fun to be paranoid! You get so many new potential enemies. :) Jerry Fletcher ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 23:56, schrieb Fred Drake: > While I'd hate to make XML processing more painful than it often is, there's > no injunction not to be reasonable. Security concerns and resource limits > are cross-cutting concerns, so it's not wrong to provide safe defaults. > > Doing so *will* be back

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Donald Stufft
On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: > > It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A > > single 1 kB XML document can kill virtually any machine, even servers > > with more than hundred GB RAM. > > > > > Assuming an attacker can inject arbi

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Wed, 20 Feb 2013 22:55:57 +0100 Christian Heimes wrote: > Am 20.02.2013 21:17, schrieb Maciej Fijalkowski: > > On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes > > wrote: > >> Am 20.02.2013 17:25, schrieb Benjamin Peterson: > >>> Are these going to become patches for Python, too? > >> > >> I

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 23:45, schrieb R. David Murray: > I don't believe it does. The DTD URL is, if I remember correctly, > specified as an identifier. The fact that you can often also download the > DTD from the location specified by the identifier is a secondary effect. > > But, it's been a *long* tim

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Carl Meyer
On 02/20/2013 03:35 PM, Greg Ewing wrote: > Carl Meyer wrote: >> An XML parser that follows the XML standard is never safe to expose to >> untrusted input. > > Does the XML standard really mandate that a conforming parser > must blindly download any DTD URL given to it from the real > live interne

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Fred Drake
On Wed, Feb 20, 2013 at 5:45 PM, R. David Murray wrote: > (Wikipedia says: "Programs for reading documents may not be required to > read the external subset.", which would seem to confirm that.) Validating parsers are required to read the external subset; this doesn't apply to the parsers distrib

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread R. David Murray
On Thu, 21 Feb 2013 11:35:23 +1300, Greg Ewing wrote: > Carl Meyer wrote: > > An XML parser that follows the XML standard is never safe to expose to > > untrusted input. > > Does the XML standard really mandate that a conforming parser > must blindly download any DTD URL given to it from the rea

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Greg Ewing
Carl Meyer wrote: An XML parser that follows the XML standard is never safe to expose to untrusted input. Does the XML standard really mandate that a conforming parser must blindly download any DTD URL given to it from the real live internet? Somehow I doubt that. -- Greg _

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 22:02, schrieb Carl Meyer: > Also, despite the title of this thread, the vulnerabilities include > fetching of external DTDs and entities (per standard), which opens up > attacks that are worse than just denial-of-service. In our initial > Django release advisory we carelessly lumped

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 21:17, schrieb Maciej Fijalkowski: > On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes > wrote: >> Am 20.02.2013 17:25, schrieb Benjamin Peterson: >>> Are these going to become patches for Python, too? >> >> I'm working on it. The patches need to be discussed as they break >> backwa

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Carl Meyer
On 02/20/2013 01:53 PM, Skip Montanaro wrote: >> That's not very good. XML parsers are supposed to parse XML according >> to standards. Is the goal to have them actually do that, or just >> address DDOS issues? > > Having read through Christian's mail and several of his references, it > seems to m

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Skip Montanaro
> > I'm working on it. The patches need to be discussed as they break > > backward compatibility and AFAIK XML standards, too. > > That's not very good. XML parsers are supposed to parse XML according > to standards. Is the goal to have them actually do that, or just > address DDOS issues? Having

Re: [Python-Dev] [Python-checkins] cpython (3.3): Rebuild importlib.h after the changes introduced in 0f65bf6063ca.

2013-02-20 Thread Eric Snow
On Wed, Feb 20, 2013 at 1:16 PM, ezio.melotti wrote: > http://hg.python.org/cpython/rev/9d00c79b27e1 > changeset: 82280:9d00c79b27e1 > branch: 3.3 > parent: 82278:96b4acb253f8 > user:Ezio Melotti > date:Wed Feb 20 21:42:46 2013 +0200 > summary: > Rebuild importlib.h

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Maciej Fijalkowski
On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes wrote: > Am 20.02.2013 17:25, schrieb Benjamin Peterson: >> Are these going to become patches for Python, too? > > I'm working on it. The patches need to be discussed as they break > backward compatibility and AFAIK XML standards, too. That's not

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 17:25, schrieb Benjamin Peterson: > Are these going to become patches for Python, too? I'm working on it. The patches need to be discussed as they break backward compatibility and AFAIK XML standards, too. ___ Python-Dev mailing list Pyth

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Benjamin Peterson
2013/2/19 Christian Heimes : > Hello, > > in August 2012 I found a DoS vulnerability in expat and XML libraries in > Python's standard library. Since then I have found several more issues. > I have been working on fixes ever since. > > The README of https://pypi.python.org/pypi/defusedxml contains

[Python-Dev] The last Python 2.6 security release

2013-02-20 Thread Barry Warsaw
I just updated PEP 361 (Python 2.6 release schedule). Python 2.6 is in security maintenance, source only release mode. Official support for Python 2.6 expires on October 1 2013, and I would like to do one last release[1], i.e. 2.6.9 as close to that date as possible. I know of issue 16248, but i

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread M.-A. Lemburg
On 20.02.2013 00:16, Daniel Holth wrote: > On Tue, Feb 19, 2013 at 5:10 PM, M.-A. Lemburg wrote: > >> On 19.02.2013 23:01, Daniel Holth wrote: >>> On Tue, Feb 19, 2013 at 4:34 PM, M.-A. Lemburg wrote: >>> On 19.02.2013 14:40, Nick Coghlan wrote: > On Tue, Feb 19, 2013 at 11:23 PM, M.-A.

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread Oleg Broytman
On Wed, Feb 20, 2013 at 10:54:06AM +0100, Antoine Pitrou wrote: > Le Wed, 20 Feb 2013 13:09:13 +0400, > Oleg Broytman a ??crit : > > On Wed, Feb 20, 2013 at 08:23:19AM +0100, Antoine Pitrou > > wrote: > > > On Tue, 19 Feb 2013 20:37:36 -0800 > > > Eli Bendersky wrote: > > > > On Tue, Feb 19, 2

Re: [Python-Dev] PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Antoine Pitrou
Le Tue, 19 Feb 2013 19:54:21 -0500, Fred Drake a écrit : > On Tue, Feb 19, 2013 at 6:19 PM, Donald Stufft > wrote: > > Let's not add anything to the stdlib till it has real world usage. > > Doing otherwise is putting the cart before the horse. > > I'd posit that anything successful will no longe

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread Antoine Pitrou
Le Wed, 20 Feb 2013 13:09:13 +0400, Oleg Broytman a écrit : > On Wed, Feb 20, 2013 at 08:23:19AM +0100, Antoine Pitrou > wrote: > > On Tue, 19 Feb 2013 20:37:36 -0800 > > Eli Bendersky wrote: > > > On Tue, Feb 19, 2013 at 10:50 AM, Oleg Broytman > > > wrote: Oleg, lately I have the feeling you'

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread M.-A. Lemburg
On 20.02.2013 03:37, Paul Moore wrote: > On 20 February 2013 00:54, Fred Drake wrote: >> I'd posit that anything successful will no longer need to be added to >> the standard library, to boot. Packaging hasn't done well there. > > distlib may be the exception, though. Packaging tools are somewha

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread Oleg Broytman
On Wed, Feb 20, 2013 at 08:23:19AM +0100, Antoine Pitrou wrote: > On Tue, 19 Feb 2013 20:37:36 -0800 > Eli Bendersky wrote: > > On Tue, Feb 19, 2013 at 10:50 AM, Oleg Broytman wrote: > > Oleg, lately I have the feeling you're getting too automatic with this > > template response. > > +1. This

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread rahul garg
> Date: Tue, 19 Feb 2013 12:48:02 -0600 > Subject: Re: [Python-Dev] Built with VS2012 Express for desktop > From: br...@python.org > To: rahulg...@live.ca > CC: python-dev@python.org > > On Tue, Feb 19, 2013 at 12:37 PM, rahul garg wrote: > > Hi. > > > > I downloaded Python 3.3 source, opened up

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Chris Jerdonek
> On Wednesday, February 20, 2013 at 2:48 AM, Chris Jerdonek wrote: > > I meant that bringing distlib into http://hg.python.org/cpython/ would > give it more visibility to core devs and others that already keep an > eye on python-checkins (the mailing list). And I think seeing the > Sphinx-processe

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Paul Moore
On 20 February 2013 04:07, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/19/2013 09:37 PM, Paul Moore wrote: >> On 20 February 2013 00:54, Fred Drake wrote: >>> I'd posit that anything successful will no longer need to be added >>> to the standard library, to boot