[Python-Dev] CVE tracking
Hello! Does someone systematically track the CVE vulnerability list? Ideally, Python security officers would have close collaboration with whoever manages CVE (like distribution security officers do), so that * every CVE issue would have a corresponding ticket on Python bug tracker (perhaps the process can be automated to some degree?) * that ticket would be referred to in CVE vulnerability page "References" section (see e.g. http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2315 , that does not have a corresponding Python bug tracker link) * all CVE issues would be listed in http://www.python.org/news/security/ with corresponding information about when the fix has been or will be commited and which upcoming or past release incorporates it. Some relevant links: http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python http://secunia.com/advisories/product/14172/?task=advisories ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] CVE tracking
I created a script that parses the
http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python
Python-related CVE list and classifies the CVEs as follows:
* "ok" -- CVE has references to bugs.python.org
* "warnings" -- CVE has references to Python SVN revisions
or an issue in bugs.python.org refers to it (i.e. the probelm is
probably fixed, but the CVE should really be updated to link
to the issue that is probably listed in bugs.python.org)
* "errors" -- CVE does have no references to Python issues or SVN
nor does any issue in bugs.python.org have references to the CVE id
The script is at http://dpaste.com/hold/92930/
The results are at http://dpaste.com/hold/92929/
There were 35 errors, 8 warnings, 5 CVEs were OK.
In an ideal world, the references would be symmetric, i.e. every
Python-related CVE would have references to one or more issues in
bugs.python.org and these issues would also refer back to the CVE id.
###
As for the rmtree problem that Gisle Aas raised, this seems to apply
as of Python 2.6:
---
# emulate removing /etc
$ sudo cp -a /etc /root/etc/
$ sudo python2.6
>>> for i in xrange(0, 5):
... with open("/root/etc/" + str(i), "w") as f:
... f.write("0")
...
$ ls /root/etc > orig_list.txt
$ mkdir /tmp/attack
$ cp -a /root/etc/* /tmp/attack
$ sudo python2.6
>>> from shutil import rmtree
>>> rmtree('/tmp/attack')
>>> # press ctrl-z to suspend execution
^Z
[1]+ Stopped sudo python2.6
$ mv /tmp/attack /tmp/dummy; ln -s /root/etc /tmp/attack
$ fg
sudo python2.6
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.6/shutil.py", line 225, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/usr/local/lib/python2.6/shutil.py", line 223, in rmtree
os.rmdir(path)
OSError: [Errno 20] Not a directory: '/tmp/attack'
$ ls /root/etc > new_list.txt
$ diff -q orig_list.txt new_list.txt
Files orig_list.txt and new_list.txt differ
---
If the attack weren't possible, the lists would not differ.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] CVE tracking
When I looked through that list a week or so ago, I noticed that some issues were obviously related to the Python distribution itself, but others were appeared to be Python application problems. I looked through the list now and weeded out irrelevant CVEs (by putting them into the ignore list in the script). Also, now the output has descriptions of the CVEs as well, so it's more readable. Improved output: http://dpaste.com/hold/93386/ Improved script (with a proper IGNORED_LIST): http://dpaste.com/hold/93388/ The results are much better: 5 OK's, 8 WARNings, 7 ERRORs. Most of the errors are from 2007 or before, the only error from 2008 is an obscure Tools/faqwiz/move-faqwiz.sh-related one. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] __import__ problems
Python programmers need to dynamically load submodules instead of
top-level modules -- given a string with module hierarchy, e.g.
'foo.bar.baz', access to the tail module 'baz' is required instead
of 'foo'.
Currently, the common hack for that is to use
modname = 'foo.bar.baz' mod = __import__(modname, {}, {}, [''])
This, however, is indeed an undocumented hack and, what's worse,
causes 'baz' to be imported twice, as 'baz' and 'baz.' (with tail
dot). The problem is reported in [1] and the idiom pops up in about
2000 (sic!) lines in Google Code Search [2].
There at least two workarounds:
* the getattr approach documented in [3]
* use __import__(modname, {}, {}, [modname.rsplit(".", 1)[-1]])
As both of them are clumsy and inefficient, I created a simple patch
for __import__ [4] that adds yet another argument, 'submodule'
(maybe 'tail_module' would have been more appropriate) that caters
for that particular use case:
__import__('foo.bar.baz') # submodule=False by default
__import__('foo.bar.baz', submodule=True)
__import__('foo.bar.baz', fromlist=['baz'])
---
While I was doing that, I noticed that the import_module_level()
function that does the gruntwork behind __import__ does not entirely
match the documentation [3].
Namely, [3] states "the statement from spam.ham import eggs results in
__import__('spam.ham', globals(), locals(), ['eggs'], -1)."
This is incorrect:
__import__('foo.bar', globals(), locals(), ['baz'], -1)
i.e. 'bar' is imported, not 'baz' (or 'ham' and not 'eggs').
As a matter of fact, anything can be in 'fromlist' (the reason for
the API abuse seen in [2]):
__import__('foo.bar.baz', globals(), locals(),
... ['this_is_a_bug'], -1)
So, effectively, 'fromlist' is already functioning as a boolean that
indicates whether the tail or toplevel module is imported.
Proposal:
* either fix __import__ to behave as documented:
# from foo.bar import baz
>>> __import__('foo.bar', fromlist=['baz'])
# from foo.bar import baz, baq
>>> __import__('foo.bar', fromlist=['baz', 'baq'])
(,
)
and add the 'submodule' argument to support the common
__import__ use case [4],
* or if that is not feasible, retain the current boolean behaviour
and make that explicit by renaming 'fromlist' to 'submodule' (and
require the latter to be a boolean, not a list).
Too bad I couldn't come up with this before, 3.0 would have been a
perfect opportunity to get things right (one way or the other).
---
References:
[1] http://bugs.python.org/issue2090
[2] http://google.com/codesearch?hl=en&lr=&q=__import__.*%5C%5B%27%27%5C%5D
[3] http://docs.python.org/library/functions.html#__import__
[4] http://bugs.python.org/issue4438
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __import__ problems
Nick Coghlan wrote:
i.e.
"from foo.bar import baz" >
= __import__('foo.bar', globals(), locals(), ['baz'], -1)
baz = .baz
When there are multiple names being imported or an 'as' clause is
involved, I hope the reasons for doing it this way become more obvious:
"from foo.bar import baz, bob" >
= __import__('foo.bar', globals(), locals(), ['baz', 'bob'], -1)
baz = .baz
bob = .bob
"from foo.bar import baz as bob" >
= __import__('foo.bar', globals(), locals(), ['baz', 'bob'], -1)
bob = .baz
Thanks for clarifying this! I'd say the current wording of the docs is
non-obvious
in that regard and your examples would be a nice addition to them.
There's a perfectly correct approach documented as part of the
__import__ docs that you referenced in your original email. If
developers decide not to use that approach and use an undocumented hack
instead, they have no right to complain when their code doesn't work
properly.
There's a good reason for the hack -- the documented loop over components
+ getattr is both a bit ugly and inefficient (as is
fromlist=[modname.rsplit(".", 1)[-1]]). The variant proposed by Hrvoje
Niksic:
>>> __import__(modname)
>>> mod = sys.modules[modname]
looks more appealing, but comes with the drawback that sys has to be
imported for
that purpose only.
As none of the variants is really in the spirit of Python's zen and the
use case
"given a string of dotted module names, import the tail module" is
really common,
I'd still say we should go with the flow and provide people what they
need explicitly
(i.e. something in the lines of http://bugs.python.org/issue4438 ) --
and disable
the hack on the same occasion (by the way, is there a reason why both
__import__('foo', fromlist=['invalid']) and __import__('foo',
fromlist=['']) don't raise
an error as of now?). This way the hacks will be eventually fixed in all
those 2000
lines listed in Google Code.
Perhaps 3.1 and 2.7 would be an appropriate chance for that?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __import__ problems
The variant proposed by Hrvoje Niksic: >>> __import__(modname) >>> mod = sys.modules[modname] looks more appealing, but comes with the drawback that sys has to be imported for that purpose only. That is not a real drawback, as "sys" will certainly be present in the system, so the "importing" boils down to a dict lookup and a variable assignment. I meant that you have to import sys only to access sys.modules (i.e. importing sys may not be necessary otherwise). Compare mod = __import__(modname, submodule=True) with import sys __import__(modname) mod = sys.modules[modname] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __import__ problems
Nick Coghlan wrote: As Hrvoje has pointed out, 'sys' is part of the internal interpreter machinery - it's there as soon as the interpreter starts. The import call for it just grabs it out of the module cache and creates a reference to it in the current namespace. I understand that, but Explicit is better than implicit. --> non-explicit import sys and __import__(modname) to access sys.modules[modname] Simple is better than complex. --> three lines for a common use case instead of one Readability counts. --> why is sys imported in this WSGI app (that shouldn't generally touch any of the common sys.argv, sys.stdout etc)?! (500 lines later) aha, to access sys.modules There should be one-- and preferably only one --obvious way to do it. --> should I use the 3-line idiom? should I use the getattr() idiom? Ideally, __import__() should support the only obvious way. In short, the three-line version doesn't convey the intent in an obvious way and does more work than is necessary. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __import__ problems
> If __import__ was replaced with a version with NON compatible interface,
> "import x.y.z" would break.
But it is not. The proposed __import__(name, submodule=True) has
a compatible interface. All tests pass with
http://bugs.python.org/file12136/issue4438.diff .
As for the imp approach, I've alternatively implemented
imp.import_module() that imports tail modules in
http://bugs.python.org/file12147/imp_import_module.diff
(colour diff in http://dpaste.com/hold/94431/).
IMHO the functionality duplication with __import__ is ugly, but
if it is desired that __import__ does not change (even in
backwards-compatible way), so be it.
The signature and behaviour is as follows:
---
import_module(name, globals, level) -> module
Import the tail module, given dotted module hierarchy in 'name'.
'globals' only determines the package context and is not
modified. 'level' specifies whether to use absolute or relative
imports.
>>> from imp import import_module
>>> import_module('foo.bar.baz')
---
But I'm still +1 for adding 'tail'/'submodule'/'tailmodule'
argument to __import__ instead of providing an almost identical
copy in imp.import_module().
Let me know which of the approaches is desired (if any) and I'll
add tests and update docs.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __import__ problems
Brett Cannon wrote: > The old-hands on python-dev know this is where I plug my import > rewrite vaporware. It will be in 3.1, and as part of it there will be > a new API for handling direct imports. Jacob Kaplan-Moss and I have Sounds good. I can finally rest in peace :) . May I suggest that you disable the hack while you are at it as well so that people will be aware of their misdoings? > talked about Django's need for this as PyCon so I am very aware of > needing this API and it will be addressed in the simplest way possible > (heck, the __import__ API might actually become a wrapper around the > simpler API in the end). Even better (the bracketed part). > And since nothing can go into 3.0 anyway, > there is no need to rush into solving this right now. Agreed, I just wanted to get the ball rolling. Let me know if you want me to do some gruntwork (tests, documentation) once the improved implementation is taking shape. --- As for the other comments, I'll try to wrap things up: * I did get the impression from some posts that it was assumed to be somehow "my problem" -- although indeed seeing both 'foo' and 'foo.' when printing sys.modules in a popular web framework I frequently use makes me wince in discomfort, the hack is present in 2000 lines in various projects as seen in the Google Code Search. * runpy.run_module() is not the solution as it runs the module each time it is called and particularly because access to the submodule object is generally needed (again, look around in the Google Code Search results). * changing the signature of __import__ is out of question both because it would break the existing __import__ replacements and would perpetuate the wrong assumption that it should be directly used (instead of the presently missing simpler interface). --- It looks that the __import__(modname) mod = sys.modules[modname] idiom is the clear winner for the import submodule use case. I've updated http://bugs.python.org/issue4457 with proposed additions to current __import__ docs. Once the simpler interface emerges, the docs should be updated again and __import__ use should be discouraged. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
