Re: [Python-Dev] Reviving restricted mode?
Hey guys, benjamin> Even if this patch manages to plug all the holes in the benjamin> current Python, do we really want to commit our benjamin> selves to maintaining it through language evolution benjamin> which will surely introduce new subtle ways to benjamin> circumvent the guard? If it would be helpful, I am happy to maintain this as Python evolves. I've already been maintaining the PJE-inspired ctypes-based approach and monkeypatches for various Python versions for a while now. See secure.py, secure25.py, secure26.py and secure30.py in: http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util Also, my plans for world domination depend on a secure Python, so I have the necessary incentives ;p sameule> I don't have much time these days, for sure not samuele> until pycon us, to look at the proposed code. Thanks in advance if/when you get the time for this Samuele! samuele> E provides and incorporate a lot of thinking samuele> around [snip] The functions based approach I am taking is very much taken from E and inspired by an insight that Ka-Ping Yee had on Python-Dev years ago. See http://www.erights.org/elib/capability/ode/index.html for a direct parallel to the approach I've taken... guido> For Tav's benefit, I think it would be good to at guido> least add "IsRestricted" checks to guido> __subclasses__(), gi_code and gi_frame -- guido> that's a trivial patch and if he believes it's guido> enough he can create a sandbox on app engine guido> and invite people to try to break out of it... If guido> someone succeeds If someone succeeds... ...My missus might end up leaving me on account of so much crying ;p Seriously though, it's a relatively risk-free approach. The only person who stands to lose out is me if I'm wrong =) In the worst case scenario, this approach would help identify other "leak" attributes/methods -- which I'm hoping won't be found. And, in an ideal scenario, we'd have the basis for secure Python interpreter/programming... which, together with PyPy's sandboxed interpreter, would seriously rock! -- enthusiastically, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
Dearest fellow Python lovers, Could one of you please review: http://codereview.appspot.com/20051 The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter! [This patch is for Python 2.5.4 -- I can create one for the other branches too if wanted...] It turns out that the __builtins__ based restricted framework is pure genius and gives us nearly everything that's needed to secure our beloved interpreter. This patch simply closes the two holes in 2.5.x -- type.__subclasses__ and GeneratorType.gi_frame. GeneratorType.gi_code would need to be restricted in 2.6 and above. The problem with rexec and brethren have simply been that we tried to do class-based sandboxing. This approach is filled with dark exploitable corners. In contrast, a function-based approach gives us security through the simplicity of the object-capability model. Or at least I currently believe so =) Once this patch gets through onto App Engine, I'll create an app with a sandboxed framework built around it and you can all prove me wrong. But, for the sake of my relationship with the missus, I hope not ;p Please note that this patch doesn't do anything to prevent any of the various crashers in Python. It is not the intention of this patch to make the interpreter invincible against segfaults or exhaustion of resource attacks. For that, I heartily recommend taking a look at PyPy's sandboxed interpreter and/or the magic of App Engine. In the extremely remote chance that I am right -- pigs could fly, right? ;p -- and the sandboxed App Engine app turns out to be impenetrable, I would like to then get approval to simplify the current restricted execution support in the interpreter and create a modernised equivalent of the rexec module. Does that seem reasonable to you all? tav> http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util martin> What is the objective of this code? Is it a complete martin> sandbox? If not, is a complete sandbox based on martin> it available somehow for review? Martin, sorry, not yet. I'll code one once the patch gets through and release it for use/review. And if people like it, it could form the basis for the modernised rexec I mentioned above... krstic> http://radian.org/~krstic/sandbox.py Thank you Ivan for that Genshi sample! Would you be interested in working with me on the Genshi aspect for the new sandbox framework? pje> Just a question, but, if you just need a pure-python pje> restricted environment for App Engine, why not just pje> use the RestrictedPython package? I'm aware of the various Zope offerings. Performance is the answer to your question. Also, when it comes to security, I am a deep believer in simplicity. Thanks again! -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
And, here's a version for Python 2.6+ -- diffed against an svn checkout of the current python/trunk: http://codereview.appspot.com/21051/show Please review also. Cheers! -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Challenge: Please break this! (was: Reviving restricted mode)
Hey all,
As an attempt to convince everyone of the merits of my functions-based
approach to security, I've come up with a simple challenge. I've
attached it as safelite.py
The challenge is simple:
* Open a fresh Python interpreter
* Do: >>> from safelite import FileReader
* You can use FileReader to read files on your filesystem
* Now find a way to *write* to the filesystem from your interpreter
Please note that the aim of this isn't to protect Python against
crashes/segfaults or exhaustion of resources attacks, so those don't
count.
I'm keen to know your experiences even if you don't manage to write to
the filesystem -- and especially if you do!
Dinner and drinks on me for an evening -- when you are next in London
or I am in your town -- to the first person who manages to break
safelite.py and write to the filesystem.
Good luck and thanks! =)
> If you block __closure__ and __globals__ on function objects you will get a
> semblance of a private namespace. That way you might (I have not thought
> this one through like securing the interpreter for embedding) be able to get
> what you need to safely pass in Python code through the globals of the code
> being executed.
Brett, this is exactly what I do. You also need to restrict func_code.
The patch is simply for closing the other loopholes:
type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible
in a patch of 6 lines of code thanks to Python's existing restricted
framework in the interpreter.
Please review and accept =)
* http://codereview.appspot.com/20051
* http://codereview.appspot.com/21051
Thanks!
--
love, tav
plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | @tav | skype:tavespian
"""
Please try and break this.
On a fresh Python interpreter, do the following:
>>> from safelite import FileReader
You should now be able to read files as you want...
Now, please try and *write* to a file on the filesystem from within the
interpreter.
Please note that the aim of this isn't to protect Python against segfaults or
exhaustion of resources attacks, so those don't count.
Let me know or Python-Dev how your experience goes -- whether
this seems to work for you or not. Thanks!
"""
import __builtin__
import sys
from sys import _getframe as get_frame
from types import FunctionType, GeneratorType
__all__ = ['FileReader']
# --
# map funktion attribute names for python versions >= 2.6
# --
FUNCTION_PY26_ATTRS = {
'func_code': '__code__',
'func_globals': '__globals__',
'func_closure': '__closure__'
}
# --
# sekure the interpreter!
# --
def secure_python():
"""Remove insecure variables from the Python interpreter."""
from ctypes import pythonapi, POINTER, py_object
get_dict = pythonapi._PyObject_GetDictPtr
get_dict.restype = POINTER(py_object)
get_dict.argtypes = [py_object]
def dictionary_of(ob):
dptr = get_dict(ob)
if dptr and dptr.contents:
return dptr.contents.value
if sys.version_info >= (3, 0):
py_version = 2
elif sys.version_info >= (2, 6):
py_version = 1
else:
py_version = 0
for attr in FUNCTION_PY26_ATTRS.keys():
if py_version <= 1:
del dictionary_of(FunctionType)[attr]
if py_version >= 1:
del dictionary_of(FunctionType)[FUNCTION_PY26_ATTRS[attr]]
del dictionary_of(type)['__subclasses__']
del dictionary_of(GeneratorType)['gi_frame']
if py_version:
del dictionary_of(GeneratorType)['gi_code']
def secure_python_builtins():
"""Remove dangerous builtins like ``file`` and patch appropriately."""
for item in ['open', 'file', 'execfile']:
del __builtin__.__dict__[item]
def null(*args, **kwargs):
pass
import linecache
linecache.open = FileReader
import site
site.file = FileReader
__builtin__.__import__ = null
# --
# pseudo-klass-like namespase wrapper
# --
class NamespaceContext(type):
"""A Namespace Context metaclass."""
def __call__(klass, __getter):
for name, obj in __getter:
setattr(klass, name, obj)
return type.__call__(klass, __gette
Re: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode)
Hey Brett, > Ah, OK. I just quickly looked at your patches on codereview and noticed that > neither __closure__ or __globals__ have been touched. Those are already restricted by Python when __builtins__ is not the same as the standard one. > I assume you are worried about getting a hold of the code type and > constructing code objects from scratch? Right. And, func_code is restricted too so didn't need any additional support. >> Please review and accept =) > I personally don't have the time. The feedback in this email is all I can > spare. That's cool -- thanks for this much! I'm hoping someone out there has a few spare minutes. The patch is just 6 lines of code... Someone? Pretty please? =) -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode)
Woo!
victor> >>> file('0wn3d', 'w').write('w00t\n')
victor> Cool. It's a good reason to go to Pycon UK this yeak ;-)
Thank you so much Victor! Please mail/phone me when you are heading to
London and I shall honour the evening out!
Now, how about this adapted version without reload?
I could make life much easier for myself by just removing all builtins
except the fundamental types, but this is way more interesting!!
Please continue the challenge the same as before, but with the new safelite.py
steven> If you really want people to try and break it, I
steven> suggest you send it to c.l.py
Done.
--
love, tav
plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
"""
Please try and break this.
On a fresh Python interpreter, do the following:
>>> from safelite import FileReader
You should now be able to read files as you want...
Now, please try and *write* to a file on the filesystem from within the
interpreter.
Please note that the aim of this isn't to protect Python against segfaults or
exhaustion of resources attacks, so those don't count.
Let me know or Python-Dev how your experience goes -- whether
this seems to work for you or not. Thanks!
"""
import __builtin__
import sys
from sys import _getframe as get_frame
from types import FunctionType, GeneratorType
__all__ = ['FileReader']
# --
# map funktion attribute names for python versions >= 2.6
# --
FUNCTION_PY26_ATTRS = {
'func_code': '__code__',
'func_globals': '__globals__',
'func_closure': '__closure__'
}
# --
# sekure the interpreter!
# --
def secure_python():
"""Remove insecure variables from the Python interpreter."""
from ctypes import pythonapi, POINTER, py_object
get_dict = pythonapi._PyObject_GetDictPtr
get_dict.restype = POINTER(py_object)
get_dict.argtypes = [py_object]
def dictionary_of(ob):
dptr = get_dict(ob)
if dptr and dptr.contents:
return dptr.contents.value
if sys.version_info >= (3, 0):
py_version = 2
elif sys.version_info >= (2, 6):
py_version = 1
else:
py_version = 0
for attr in FUNCTION_PY26_ATTRS.keys():
if py_version <= 1:
del dictionary_of(FunctionType)[attr]
if py_version >= 1:
del dictionary_of(FunctionType)[FUNCTION_PY26_ATTRS[attr]]
del dictionary_of(type)['__subclasses__']
del dictionary_of(GeneratorType)['gi_frame']
if py_version:
del dictionary_of(GeneratorType)['gi_code']
def secure_python_builtins():
"""Remove dangerous builtins like ``file`` and patch appropriately."""
# thanks Victor Stinner!
for item in ['open', 'file', 'execfile', 'reload']:
del __builtin__.__dict__[item]
def null(*args, **kwargs):
pass
import linecache
linecache.open = FileReader
import site
site.file = FileReader
__builtin__.__import__ = null
# --
# pseudo-klass-like namespase wrapper
# --
class NamespaceContext(type):
"""A Namespace Context metaclass."""
def __call__(klass, __getter):
for name, obj in __getter:
setattr(klass, name, obj)
return type.__call__(klass, __getter)
def __str__(klass):
return 'NamespaceContext%s' % (tuple(klass.__dict__.keys()),)
def _Namespace():
__private_data = {}
def Namespace(*args, **kwargs):
"""Return a Namespace from the current scope or the given arguments."""
class NamespaceObject(tuple):
__metaclass__ = NamespaceContext
__slots__ = ()
def __new__(klass, __getter):
return tuple.__new__(klass, __getter)
ns_items = []; populate = ns_items.append
if args or kwargs:
frame = None
for arg in args:
kwargs[arg.__name__] = arg
for name, obj in kwargs.iteritems():
if isinstance(obj, FunctionType):
populate((name, staticmethod(obj)))
else:
populate((name, obj))
else:
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
> I take it back, we need to find all the trivial ones too. Agreed! > BTW Tav, you ought to create a small website for this challenge. A > blog post or wiki page would suffice. Done. http://tav.espians.com/a-challenge-to-break-python-security.html Please blog/retweet and of course, try the challenge yourselves =) -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
Hey Martin, >> The patch is a mere 6 lines of code and provides the absolute minimum >> that is needed to secure the Python interpreter! > > Unlike Guido, I'm not quite willing to your word for it. You are right. Sorry, I was a bit too enthusiastic and overstated the case. How about: "it could possibly enable a secured Python interpreter" ? > OTOH, the patch looks harmless (with minor corrections). It could > be considered a bug fix for the current set of restricted attributes Yes, and it is in that light that I would like the patch to be accepted. -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
guido> >>> class S(str): guido> ... def __eq__(self, o): print o; return 'r' == o guido> [snip] Very devious -- @eichin and Guido! You guys get the price for the cutest exploit yet -- but sadly no dinner or drinks -- that was just for the first crack -- which goes to Victor =) steve> Don't I remember the previous restricted module dying a steve> similar "death of 1,000 cuts" before it was concluded steve> to be unsafe at any height and abandoned? Steve, this isn't death by a 1,000 cuts. What's being put forward here is not a specific implementation -- but rather a specific model of security (the object capability model) -- which has been proven to be foolproof. The question here is whether Python can support that. And, my belief is that it can. Besides the really nice __eq__ hack, the other exploits so far are just an inappropriate setup of the environment -- the trick with object capability is *ensuring* that unsafe references aren't passed to untrusted code. In an earlier version of safelite, I even returned the actual file object when f.close() was called... oops! But that doesn't invalidate the model or the possibility of using it in Python. What would invalidate it is someone finding a way to bypass it completely in Python and this challenge is an attempt to see if we can find such a way. -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode)
Hey all, victor> Could you keep all versions of safelite.py? I took Steven D'Aprano's advice and added a VERSION attribute and state the latest version on http://tav.espians.com/a-challenge-to-break-python-security.html Is that okay? antoine> I guess Tav should open a restaurant :-) Hehe!! Thankfully I only offered to it to the first person *phew!* farshid> It seems like some code in safelite passes farshid> a file object to isinstance. By overriding the farshid> builtin isinstance function I can get access to frashid> the original file object and create a new one. Farshid, this is beautiful!!! Thank you -- it's very nicely done!! Do you have a website I could link to from the blog article? guido> I think in the next version Tav will have to stop guido> the sharing of __builtins__ between the supervisor guido> and the sandboxed code. There are too many guido> tricks you can play with this. Done. The common pattern that arised out of the various builtins-overriding-hacks is that "safe" code should *never* make assumptions about the state of the globals. The use of closures seems to fix this problem with an easily-auditable design pattern. -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
Hey Victor,
You definitely got to the heart of the challenge.
> f.tell.__getattribute__('func_closure')
But, have you actually run that code?
Cos that doesn't work here... sorry if I missed something...
--
love, tav
plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
guido> I can access the various class and metaclass objects
guido> easily [snip]
It would've been possible to replace __call__ on the metaclass --
which, though not a security leak by itself, could've been abused for
some fun.
I've inlined the __metaclass__ to prevent fun of this kind.
But the really tricky one so far is this hardcore hack that Paul
Cannon emailed in:
>>> from safelite import FileReader
>>> __builtins__.TypeError = type(lambda: 0)(type(compile('1', 'b', 'eval'))(2,
>>> 2, 4, 67,
>>> 'y\x08\x00t\x00\x00\x01Wn\x09\x00\x01\x01a\x00\x00n\x01\x00X|\x01\x00|\x00\x00\x83\x01\x00S',
>>> (None,), ('stuff',), ('g', 'x'), 'q', 'f', 1, ''), globals(), None,
>>> (TypeError,))
>>> try:
... FileReader('foo', 2)
... except:
... pass
...
>>> stuff.tb_frame.f_back.f_locals['open_file']('w00t', 'w').write('yaymore\n')
He explains it in detail here:
http://thepaulprog.blogspot.com/2009/02/safelite-exploit.html
It's very cool!
He uses the ``compile`` builtin to get hold of the CodeType and then
uses that to construct a nifty function with custom bytecode. Turns
out that with the bytecode one can grab traceback objects off of the
stack!!
And, from there, it's just a mere attribute access away to get hold of
traceback.tb_frame!
Paul, wisely, outlines the two possible options to remedy this:
1. Remove ``compile`` from the safe builtins
2. Take out ``tb_frame``
If compile is not present and given that func_code and gi_code are not
present -- are there other ways of getting at the CodeType? If there
aren't then getting rid of compile gives us an easy win.
If getting rid of tb_frame is the approach -- then you were right
Guido -- there are more variables which need to be restricted! But,
this just makes the patch into 7 lines of code instead of 6, so I'm
still happy =)
The only thing I can't figure out is how to get rid of the attribute
using ctypes for the safelite challenge as calling
dictionary_of(TracebackType) in safelite.py presents a very minimal
{'__doc__': None}
Also, the comments in Lib/types.py states:
# In the restricted environment, exc_info returns (None, None,
# None) Then, tb.tb_frame gives an attribute error
I can't seem to find the place in the Python source where exc_info()
behaves differently under restricted mode...
Thoughts on which of the two options is better would be very appreciated!
And thanks for the ongoing hacks guys -- this is turning out great!!
--
love, tav
plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
Daniel emailed in the exploit below and it is pretty devastating. It
takes advantage of the fact that the warnings framework in 2.6+
dynamically imports modules without being explicitly called!!
I've fixed this hole in safelite.py, but would be interested to know
if there are other non-user-initiated dynamically imported modules?
Thanks Daniel for bringing this to our attention!
On Tue, Feb 24, 2009 at 4:46 AM, Daniel (ajax) Diniz wrote:
> Here's my attempt. Uses the builtin warnings module.You might need to
> change 'args[1]' to 'args[2]' depending on your environment.
>
> I really don't like to post these to public MLs, but feel free to do
> so if you think no harm may arise from this one ;)
>
> Daniel
>
>
> from safelite import FileReader
>
> # Let's build a fake module
> warnings = __builtins__.__class__('warnings')
>
> # Fill it with deception
> warnings.default_action = "ignore"
>
> # And provide a supporting thug
> def __import__(*args):
> try:
> print "How nice:\n", args[1].keys()
> global sys
> sys = args[1]['sys']
>
> except Exception, v:
> print "Exception:", v
> return warnings
>
> # Put the bogus module at the doorstep...
> __builtins__.warnings = warnings
>
> # and have the thug replacing the doorman
> __builtins__.__import__ = __import__
>
> # An unsuspecting costumer passes by...
> FileReader('safelite.py').seek(1.1)
>
> # ... and is brutally mugged :)
> print sys
> print dir(sys)
>
--
love, tav
plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
antoine> You'd better make __builtins__ read-only, it will antoine> plug a whole class of attacks like this. I tried to put this off as long as I could to try and unearth interesting attacks. But unfortunately I couldn't figure out a way to fix the warnings approach used by Daniel without doing this -- so from v7 __builtins__ isn't shared any more. The good thing is that we won't have more of the __builtins__ class of attacks -- the flip side is that we might be closing the door on discovering some really interesting gems... andrew> I can look up the stack frames and get andrew> "open_file", which I can then use for whatever I want. Ehm, thanks for taking the time to implement that Andrew. But the challenge was about doing `from safelite import FileReader`. I specifically stated that form over the openly exploitable `import safelite`... so, sorry =( You have to remember that this isn't the way that this code will actually be used in practise. This is just a challenge to see if the model holds... -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
Hey Nick, > Given the context manager hack I just sent you (similar in spirit to > Paul's, just using a context manager's __exit__() method to get hold of > the traceback instead of hacked bytecode) Thanks for this -- very cool! > I think we can safely say that tb_frame has to go. I've fixed this in v8 -- got a website that I can link to for the blog? And instead of trying to make tb_frame go away, I'd like to add the following to my proposed patch of RESTRICTED attributes: * f_code * f_builtins * f_globals * f_locals That seems to do the trick... -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python jail: whitelist vs blacklist
Hey Victor, > Today it's clear that tav's jail is broken. Forgive me as I'm sleep deprived, but no =) > Many hackers proved how to break it. > Fixing each hole is maybe not the good solution. The aim of this challenge has been to: 1. Validate the functions-based approach 2. Verify if the proposed set of new attribute RESTRICTIONs are enough As such, it has been important to ensure that we have as large an attack surface as possible. And given how trivial it has been to fix the bugs, the functions-based approach seems to be holding up =) And as for the attributes needing to be restricted, we've discovered that we need to restrict the f_* attributes of frameobject along with the initial type.__subclasses__, gi_code and gi_frame. > A better approach would be to create a new world (namespace) from an empty > namespace and then add our "secure" functions/types using strong validations > (using a whitelist instead of a blacklist). Sure -- you are absolutely right about using a whitelist approach. safelite.py is just for the challenge... to demonstrate that the functions-based approach could possibly lead to securing the Python interpreter and to verify that we've restricted the necessary attributes. Once the patch gets accepted, we can start creating a fresh world built up from an object capability base =) > In CPython, we may use proxies on anything to check all operations. > jail -- validations --> real world > jail <-- proxy objects -- real world Ehm, I'd strongly discourage any approaches using proxies. The performance penalties will just be insane. If you really want one though -- check out Zope proxy. It already implements this quite well and you can use it today! =) -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Challenge: Please break this! [Now with blog post]
Hey Andrew & Victor, tav> But the challenge was about doing `from safelite import FileReader`. Andrew> Though it doesn't say so on the first post on this thread Andrew> nor your page at Andrew> http://tav.espians.com/a-challenge-to-break-python-security.html Sorry, perhaps I should have clearly instructed NOT to do `import safelite`. I thought the following text would have been clear enough: Open a fresh Python interpreter and do: >>> from safelite import FileReader But I can see how that might not have been enough. I'll amend the second line to: >>> from safelite import FileReader # AND NOT ``import safelite`` Would that be clearer? Andrew> I haven't been watching this discussion closely Andrew> and I can't find mention of this Sorry, the discussion has been meandering. It started on: * http://code.google.com/p/googleappengine/issues/detail?id=671 Then went onto: * http://codereview.appspot.com/21043/show And finally ended up here on Python-Dev to get approval for the patch. And in the process of getting the approval, the safelite challenge started... Andrew> is the goal to support only 2.x or also support Andrew> Python 3? Your model seems to assume 2.x Andrew> only, and there may be 3.x attacks that aren't Andrew> considered in the challenge. The model is suited for both 2.x and 3.x. The safelite test on the other hand is targeted at 2.x (starting from 2.5). You have a point with regards to there being 3.x attacks being out there. Perhaps we can get a 3.x compatible version of safelite out? tav> And instead of trying to make tb_frame go away, I'd like tav> to add the following to my proposed patch of tav> RESTRICTED attributes [snip] Andrew> which of course would make the above no longer work. The challenge helped identify those attributes. And will hopefully identify any other "leak" attributes which may still be there. Andrew> A goal is to use this in App Engine, yes? Yes. Andrew> Which uses cgitb to report errors? Which needs these Andrew> restricted frame attributes to report the values of Andrew> variables when the error occurred? The frame attributes will still be accessible by cgitb. Python's existing IsRestricted framework in the interpreter will take care of that. It's genius!! Victor> We should be able to restore the original environment. Victor> Example: Victor> jail(evil_func) # called in the jail Victor> # unsafe environment with __subclasses__, f_code, etc. Hmz, Python's IsRestricted already enables this. I think it's time that I wrote some documentation that Martin wanted explaining Python's already existing IsRestricted support. Besides a few lines here and there, there seems to be no comprehensive documentation on this that I can find anywhere. And I think this may be the cause of much confusion? Would that help? -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Paving the Way to Securing the Python Interpreter [Detailed Summary]
Dear fellow Python lovers, I have written up a detailed summary on: http://tav.espians.com/paving-the-way-to-securing-the-python-interpreter.html Please read it. In the article I give the rationale for my patch, which I've updated at: * http://codereview.appspot.com/20051/show Please review it. Consider it as simply a bug fix for the current set of restricted attributes in Python. In the article I also provide some much needed documentation on Python's existing restricted execution framework. Martin asked for this. Fell free to adapt the documentation into Doc/ articles. You can find the reStructuredText source of the article at http://github.com/tav/blog/tree/master Many thanks to everyone who took part in the challenge -- it was very informative and fun! Please let me know what else I need to do to get the patch accepted. Thanks! -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Ruby-style Blocks in Python [Pseudo-PEP]
Hey all, Apologies for bringing up an old issue, but I think I've worked out a Pythonic syntax for doing Ruby-style blocks. The short of it is: with employees.select do (emp): if emp.salary > developer.salary: return fireEmployee(emp) else: return extendContract(emp) I explain in detail in this blog article: http://tav.espians.com/ruby-style-blocks-in-python.html It covers everything from why these are useful to a proposal of how the new ``do`` statement and __do__ function could work. Let me know what you think. My apologies if I've missed something obvious. Thanks! -- love, tav plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] CPython in the web browser under Native Client
Hey Mark,
> http://lackingrhoticity.blogspot.com/2009/06/python-standard-library-in-native.html
Really glad to see that you carried on with this -- great work!!
I guess the elders will simply say that it's history repeating itself,
but Grails 2.0 is looking promising! We can finally give that upstart
Javascript a run for its money (despite ES5 looking nice...) -- not to
mention Google Wave/Opera Unite/etc.
Now, the real blocker is accessing the DOM from Python. I haven't
looked into the source code deeply, but perhaps Kroll could be of
help:
* http://github.com/jhaynie/kroll/tree/master
I was able to do the following with Titanium Desktop (which uses it)
to access the DOM from Python:
def hello(s):
document.getElementById('foo').innerHTML = s
Hello World
hello("tav")
More relevant links:
* http://www.appcelerator.com/
* http://github.com/appcelerator/
Anyways, I'm sure others might have a better idea of integrating
Python and the DOM.
Well done again!
--
love, tav
plex:espians/tav | t...@espians.com | +44 (0) 7809 569 369
http://tav.espians.com | http://twitter.com/tav | skype:tavespian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] object capability; func_closure; __subclasses__
rehi all,
I have been looking at the object capability + Python discussions for
a while now, and was wondering what the use cases for
``object.__subclasses__`` and ``FunctionType.func_closure`` were?
I don't see __subclasses__ used anywhere in the standard library. And
whilst I can see exposing func_closure as being useful in terms of
"cloning/modifying" an existing function, isn't it possible to do that
without making it introspectable?
Years ago, Ka-Ping Yee pointed out:
http://mail.python.org/pipermail/python-dev/2003-March/034284.html
Derived from this we get:
# capability.py functions
def Namespace(*args, **kwargs):
for arg in args:
kwargs[arg.__name__] = arg
def get(key):
return kwargs.get(key)
return get
class Getter(object):
def __init__(self, getter):
self.getter = getter
def __repr__(self):
return self.getter('__repr__') or object.__repr__(self)
def __getattr__(self, attr):
return self.getter(attr)
# io.py module
def FileReader(name):
file = open(name, 'r')
def __repr__():
return '' % name
def read(bufsize=-1):
return file.read(bufsize)
def close():
return file.close()
return Getter(Namespace(__repr__, read, close))
Now, a process A -- which has full access to all objects -- can do:
>>> motd = FileReader('/etc/motd')
And pass it to "process B" operating in a limited scope, which can then call:
>>> motd.read()
>>> motd.close()
But not:
>>> motd = type(motd)(motd.name, 'w')
which would have been possible *had* motd been created as a ``file``
type by calling: ``open('/etc/motd', 'r')``.
Now, there are probably a million holes in this approach, but as long
as process B's __import__ is sanitised and it operates in a "limited"
scope with regards to references to other functionality, this seems to
be relatively secure.
However, this is where __subclasses__ and func_closure get in the way.
With object.__subclasses__ (as Brett points out), all defined
classes/types are available -- including the ``file`` type we were
trying to deny process B access to! Is it necessary to expose this
attribute publically?
And, likewise with func_closure, one can do
motd.read.func_closure[0].cell_contents and get hold of the original
``file`` object. Is it absolutely necessary to expose func_closure in
this way?
Now, whilst probably wrong, I can see myself being able to create a
minimal object capability system in pure python if those 2 "features"
disappeared. Am I missing something obvious that prevents me from
doing that?
Can we get rid of them for Python 2.6? Or even 2.5.2? Is anyone
besides PJE actually using them? ;p
Thanks in advance for your thoughts.
--
love, tav
founder and ceo, esp metanational llp
plex:espians/tav | [EMAIL PROTECTED] | +44 (0) 7809 569 369
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] object capability; func_closure; __subclasses__
> You know, I find it particularly interesting that, as far as I can > tell, nobody who proposes making changes to the Python language to > add security, ever seems to offer any comparison or contrast of their > approaches to Zope's -- which doesn't require any changes to the language. :) Whilst it is definitely possible to build up a object capability system with a pruned down version of Zope 3's proxy + checker mechanism, it ends up in a system which is quite performance intensive. All those proxies being wrapped/unwrapped/checked... In contrast, the mechanism I am looking at here simply requires *moving* just 2 attributes *out* of the core builtins... Quite cheap, simple and effective, no? > Well, you're missing a simpler approach to protecting functions, > anyway. The '__call__' attribute of functions is still callable, but > doesn't provide any access to func_closure, func_code, etc. I > believe this trick also works for bound method objects. Whilst that would have been a nice trick, what about __call__.__self__ ? Or, setting __call__.__doc__ ? > I suspect that you could also use ctypes to remove or alter the > type.__subclasses__ member, though I suppose you might not consider > that to be "pure" Python any more. However, if you use a definition > of pure that allows for stdlib modules, then perhaps it works. :) Ah, thanks! Will look into that. > I wouldn't object (no pun intended) to moving the type.__subclasses__ > method to say, the 'new' or 'gc' modules, since you wouldn't want to > make those available to restricted code, but then they'd still be > available for folks who need/want them. 'gc' has similar > capabilities (again no pun intended) anyway. Ah, that's a great idea! > However, ISTM that this should be a 3.0 change rather than a 2.x one, > even so. With regard to the func_closure thing, I'd actually like to > make it *writable* as well as readable, and I don't mean just to > change the contents of the cells. But, since you can use .__call__ > to make a capability without access to func_closure, it doesn't seem > like you really need to remove func_closure. I don't object to making func_closure writable either. In fact, as someone who has been following your work on generic functions from way before RuleDispatch, I really want to see PEP 3124 in 3.0 But, all I am asking for is to not expose func_closure (and perhaps some of the other func_*) as members of FunctionType -- isn't it possible to add functionality to the ``new`` module which would allow one to read/write func_closure? -- love, tav founder and ceo, esp metanational llp plex:espians/tav | [EMAIL PROTECTED] | +44 (0) 7809 569 369 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] object capability; func_closure; __subclasses__
> Well, there's no __self__ in 2.3 or 2.4; I guess that was added in 2.5. Darn. anyone know *why* it was added? > >Or, setting __call__.__doc__ ? > > What does that do? ah, i just wanted a way of providing documentation, and __call__'s __doc__ isn't writable... > If it works, you could probably do the same thing to remove __call__.__self__. will look into that too... > In 3.0, I don't mind if the access method moves, I just want to keep > the access. OTOH, I don't really care about __call__.__self__, since > I got along fine without it in 2.3/2.4 and didn't know it had been > added in 2.5. :) w00p! so, suggestions as to how one can go about getting those 2 access methods moved? -- thanks, tav founder and ceo, esp metanational llp plex:espians/tav | [EMAIL PROTECTED] | +44 (0) 7809 569 369 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] object capability; func_closure; __subclasses__
I love you PJE! Thank you! =) On 6/28/07, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 05:23 PM 6/28/2007 +0100, tav wrote: > >Any pointers on removing members via ctypes front? > > > >Whilst I can understand even the most obscure aspects of your python > >code fine, I'm not familiar with C/ctypes... > > What you want is to get access to the type's real dictionary, not the > proxy. Then you can just delete '__subclasses__' from the dictionary > using Python code. Here's some code that does the trick: > > from ctypes import pythonapi, POINTER, py_object > > getdict = pythonapi._PyObject_GetDictPtr > getdict.restype = POINTER(py_object) > getdict.argtypes = [py_object] > > def dictionary_of(ob): > dptr = getdict(ob) > if dptr and dptr.contents: > return dptr.contents.value > > 'dictionary_of' returns either a dictionary object, or None if the > object has no dictionary. You can then simply delete any unwanted > contents. However, you should *never use this* to assign __special__ > methods, as Python will not change the type slots correctly. Heck, > you should probably never use this, period. :) Usage example: > >print "before", type.__subclasses__ >del dictionary_of(type)['__subclasses__'] >print "after", type.__subclasses__ > > This will print something like: > >before >after >Traceback (most recent call last): > File "ctypes_dicto.py", line 14, in >print "after", type.__subclasses__ >AttributeError: type object 'type' has no attribute '__subclasses__' > > et voila. > > You should also be able to delete unwanted function type attributes like > this:: > >from types import FunctionType >del dictionary_of(FunctionType)['func_closure'] >del dictionary_of(FunctionType)['func_code'] > > Of course, don't blame me if any of this code fries your computer and > gives you a disease, doesn't work with new versions of Python, etc. > etc. It works for me on Windows and Linux with Python 2.3, 2.4 and > 2.5. It may also work with 3.0, but remember that func_* attributes > have different names there. > > -- love, tav founder and ceo, esp metanational llp plex:espians/tav | [EMAIL PROTECTED] | +44 (0) 7809 569 369 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
