Re: [Python-Dev] ImportWarning flood
Ralf W. Grosse-Kunstleve wrote: > --- Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > >>I think it is safe to say that Twisted is more widely used than anything >>Google has yet released. Twisted also has a reasonably plausible >>technical reason to dislike this change. Google has a bunch of engineers >>who, apparently, cannot remember to create an empty __init__.py file in >>some directories sometimes. > > > Simply adding a note to the ImportError message would solve this problem "just > in time": > > import mypackage.foo > > Traceback (most recent call last): > File "", line 1, in ? > ImportError: No module named mypackage.foo > Note that subdirectories are searched for imports only if they contain an > __init__.py file: http://www.python.org/doc/essays/packages.html > Yeah, that'll really help the end-user whose sys admin has just upgraded to 2.5, won't it? regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Guido van Rossum wrote: > I think we need a PEP for const/static/only/cached/precomputed or > whatever people like to call it. fredrik's got a micro pep at http://online.effbot.org > Once we have (say) static, I think making the case expressions static > by default would still cover all useful cases, and would allow us to > diagnose duplicate cases reliably (which the if/elif chain semantics > don't allow IIUC). Making case expressions default static would be very surprising to users because of the restrictions placed by static. For instance 'case in a', will not support containers which have a custom __contains__ method. It will also not support containers like lists, and sets because they are mutable. IMHO this doesn't feel very pythonic. Instead if we redefine the goal of the switch statement to be 'ease of expression' rather than 'optimization', then it can just be used as a concise alternative to if-elif chains, and we can make 'case in a' work with all containers where a regular 'in' statement works *AND* still give the possibility of fast lookup when the programmer wants, using explicit static. I feel programmer expressivity is more important, and default static case expressions looks like premature optimization. Regards Sreeram signature.asc Description: OpenPGP digital signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
Le mardi 27 juin 2006 05:38, Phillip J. Eby a écrit : > At 05:16 PM 6/26/2006 -0700, Martin Maly wrote: > > >>> class D(object): > > > >... def getclass(self): > >... print "D.getclass" > >... return C > >... __class__ = property(getclass) > >... > > > > >>> isinstance(D(), D) > > > >True > > > > >>> isinstance(D(), C) > > > >D.getclass > >True > > > >isinstance in this case returns True to both C and D test. I would expect > > > >to see the __class__ property being called in both cases and get: > > >>> isinstance(D(), D) > > > >D.getclass > >False > > > >but that's not the case for some reason. > > That's because isinstance checks type(D()) and finds it equal to D -- this > shortcuts the process. > > > It seems that the __class__ is only accessed in some cases, but not > > always, leading to what I think is a semantic inconsistency. > > It's not inconsistent - isinstance() checks __class__ in *addition* to > type() in order to allow proxying tricks like lying about your > __class__. It therefore returns true if either your real type *or* your > __class__ matches, and as you can see, the real type is checked first. > > >class E(object): > > def getbases(self): > > print "E.getbases" > > return () > > __bases__ = property(getbases) > > > >class C(object): > > def getbases(self): > > print "C.getbases" > > return (E,) # C() claims: "E is my base > > class" __bases__ = property(getbases) > > > >class D(object): > > def getclass(self): > > print "D.getclass" > > return C() # D() claims: "C() is my > > __class__" __class__ = property(getclass) > > > > > >class F(object): pass > > > > > >print "Test 1" > >print isinstance(D(), E()) # testing against E() instance > >print "Test 2" > >print isinstance(D(), E)# testing against E class > > > >The output here is: > > > >Test 1 > >E.getbases > >D.getclass > >C.getbases > >False > > > >Test 2 > >D.getclass > >False > > > >In the 2nd test, D.getclass is called to get the __class__ of D(), which > >returns C() instance. At this point I would expect that C.getbases gets > >called as __bases__ are retrieved, which would return tuple consisting of > >E and ultimately produce True result. > > As it happens, this is due to the fact that E is a type, while E() is > not. There's an optimization in the isinstance() machinery that simply > checks to see if D().__class__ is a subtype of E. That's where your > experiment fails. > > I'm not sure whether this behavior should be considered correct or not. > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/maric%40aristote.info Doesn't seems to be just related to isinstance implementation, furthermore, it is very surprising that old-style and new style classes behave exactly the opposite way. In [2]: class a(object) : ...: __class__ = 0 ...: ...: In [3]: a.__class__ Out[3]: In [4]: a().__class__ Out[4]: 0 In [7]: class a : ...: __class__ = 0 ...: ...: In [8]: a.__class__ Out[8]: 0 In [9]: a().__class__ Out[9]: -- _ Maric Michaud _ Aristote - www.aristote.info 3 place des tapis 69004 Lyon Tel: +33 426 880 097 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
> PEP 3103, When to Freeze the Dispatch Dict/Option 1 2 things resonated with me for Raymond's proposal and the follow up: - It seemed agnostic to almost all of the independently contentious issues. - "is defined tightly enough to allow room for growth and elaboration over time" [Raymond]. In particular it left room for const/static/only/cached/etc to come along later. I think its worth acknowledging this in the PEP. Is nothing better than something in this case ? I don't know. > I think we need a PEP for const/static/only/cached/precomputed or > whatever people like to call it. > > Once we have (say) static, I think making the case expressions static > by default would still cover all useful cases, and would allow us to > diagnose duplicate cases reliably (which the if/elif chain semantics > don't allow IIUC). If the expectation is that static/const will evolve as a sibling pep, does this not make Raymond's suggestion any more appealing, even a little ? Is it unacceptable - or impractical - to break the addition of switch to python in two (minor version separated) steps ? Robin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 328 and PEP 338, redux
Mitch Chapman [1] tripped over the fact that relative imports don't like main
modules because __name__ doesn't contain any package hierarchy information.
It occurred to me that a slight modification to PEP 338 might solve the
problem fairly cleanly: instead of simply setting __name__ to '__main__' for a
module in a package, the -m switch could prepend the package name so that
relative imports can work correctly.
Inside the module, the test for "am I the main module" would need to be
"__name__.endswith('__main__')" instead of "__name__ == '__main__'", but other
than that, there should be very little impact.
By sticking the main module into sys.modules under two different names (once
with the package prefix and once without), runpy could even take care of
ensuring that the following invariant held:
import __main__
by_name = __import__(__main__.__name__)
assert by_name is __main__
The behaviour of top level modules would be unaffected, since they won't have
a package prefix.
Cheers,
Nick.
[1] http://www.python.org/sf/1510172
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
Phillip J. Eby wrote: >> It seems that the __class__ is only accessed in some cases, but not >> always, leading to what I think is a semantic inconsistency. > > It's not inconsistent - isinstance() checks __class__ in *addition* to > type() in order to allow proxying tricks like lying about your > __class__. It therefore returns true if either your real type *or* your > __class__ matches, and as you can see, the real type is checked first. This is not the original rationale, though: the check for a __class__ attribute on non-instance objects was introduced in r13520, to support ExtensionClasses. I never fully understood ExtensionClasses, but I believe they were not based on proxying tricks. Instead, they were an early version of new-style classes. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
Martin Maly wrote: > Thanks for the response. The code snippet I sent deals with new style > classes only so I assume that in some cases isinstance falls back to > old-style-like handling which then asks for __bases__ and __class__ > etc, possibly incorrectly so on new style classes. Again, I believe this is all included for ExtensionClasses: it looks for __class__ on the object if the type check fails, so that an ExtensionClass could be actually a class derived from the C type. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Guido van Rossum wrote:
> I've written a new PEP, summarizing (my reaction to) the recent
> discussion on adding a switch statement. While I have my preferences,
> I'm trying to do various alternatives justice in the descriptions. The
> PEP also introduces some standard terminology that may be helpful in
> future discussions. I'm putting this in the Py3k series to gives us
> extra time to decide; it's too important to rush it.
>
> http://www.python.org/dev/peps/pep-3103/
A generally nice summary, but as one of the advocates of Option 2 when it
comes to freezing the jump table, I'd like to see it given some better press :)
> Feedback (also about misrepresentation of alternatives I don't favor)
> is most welcome, either to me directly or as a followup to this post.
My preferred variant of Option 2 (calculation of the jump table on first use)
disallows function locals in the switch cases just like Option 3. The
rationale is that the locals can't be expected to remain the same across
different invocations of the function, so caching an expression that depends
on them is just as nonsensical for Option 2 as it is for Option 3 (and hence
should trigger a Syntax Error either way).
Given that variant, my reasons for preferring Option 2 over Option 3 are:
- the semantics are the same at module, class and function level
- the order of execution roughly matches the order of the source code
- it does not cause any surprises when switches are inside conditional logic
As an example of the latter kind of surprise, consider this:
def surprise(x):
do_switch = False
if do_switch:
switch x:
case sys.stderr.write("Not reachable!\n"):
pass
Option 2 won't print anything, since the switch statement is never executed,
so the jump table is never built. Option 3 (def-time calculation of the jump
table), however, will print "Not reachable!" to stderr when the function is
defined.
Now consider this small change, where the behaviour of Option 3 is not only
surprising but outright undefined:
def surprise(x):
if 0:
switch x:
case sys.stderr.write("Not reachable!\n"):
pass
The optimiser is allowed to throw away the contents of an if 0: block. This
makes no difference for Option 2 (since it never executed the case expression
in the first place), but what happens under Option 3? Is "Not reachable!"
written to stderr or not?
When it comes to the question of "where do we store the result?" for the
first-execution calculation of the jump table, my proposal is "a hidden cell
in the current namespace".
The first time the switch statement is executed, the cell object is empty, so
the jump table creation code is executed and the result stored in the cell. On
subsequent executions of the switch statement, the jump table is retrieved
directly from the cell.
For functions, the cell objects for any switch tables would be created
internally by the function object constructor based on the attributes of the
code object. So the cells would be created anew each time the function
definition is executed. These would be saved on the function object and
inserted into the local namespace under the appropriate names before the code
is executed (this is roughly the same thing that is done for closure
variables). Deleting from the namespace afterwards isn't necessary, since the
function local namespace gets thrown away anyway.
For module and class code, code execution (i.e. the exec statement) is
modified so that when a code object is flagged as requiring these hidden
cells, they are created and inserted into the namespace before the code is
executed and removed from the namespace when execution of the code is
complete. Doing it this way prevents the hidden cells from leaking into the
attribute namespace of the class or module without requiring implicit
insertion of a try-finally into the generated bytecode. This means that switch
statements will work correctly in all code executed via an exec statement.
The hidden variables would simply use the normal format for temp names
assigned by the compiler: "_[%d]". Such temporary names are already used by
the with statement and by list comprehensions.
To deal with the threading problem mentioned in the PEP, I believe it would
indeed be necessary to use double-checked locking. Fortunately Python's
execution order is well enough defined that this works as intended, and the
optimiser won't screw it up the way it can in C++. Each of the hidden cell
objects created by a function would have to contain a synchronisation lock
that was acquired before the jump table was calculated (the module level cell
objects created by exec wouldn't need the synchronisation lock). Pseudo-code
for the cell initialisation process:
if the cell is empty:
acquire the cell's lock
try:
if the cell is still empty:
build the jump table an
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Guido van Rossum wrote: > For a real example from C++, read "Double Checked Locking is Broken": > http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html > (I first heard about this from Scott Meyers at the '06 ACCU conference > in Oxford; an earlier version of his talk is also online: > http://www.aristeia.com/Papers/DDJ_Jul_Aug_2004_revised.pdf). I swear I wrote what I did about double-checked locking and Option 2 *before* reading this particular post! Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
At 10:13 PM 6/26/2006 -0700, Guido van Rossum wrote: >On 6/26/06, Greg Ewing <[EMAIL PROTECTED]> wrote: >>Phillip J. Eby wrote: >> >> > It's not inconsistent - isinstance() checks __class__ in *addition* to >> > type() in order to allow proxying tricks like lying about your >> > __class__. >> >>If this is a deliberate feature, it's a bit patchy, because >>it means the proxy can't lie about *not* being an instance >>of its real type. >> >>Perhaps Guido could clarify how much lying a proxy is >>supposed to be able to get away with? > >Sorry, I don't remember all the constraints. Read the code and weep. >This should be revisited for Py3k. The code became convoluted out of >some needs in Zope; I can't remember if it was Zope 2 or Zope 3 that >needed this (probably both) and I can't remember the specific >situation where it was needed. It was Zope 3 security proxies, although *any* proxy type benefits. The idea was to make proxy objects be able to lie about their __class__ and be believed by isinstance(). However, there was no requirement that isinstance(ob, Proxy) return False, so that's not implemented. And lying about __bases__ appears to only be allowed for things that aren't types. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
At 09:29 AM 6/27/2006 +0200, Maric Michaud wrote: >Le mardi 27 juin 2006 05:38, Phillip J. Eby a écrit : > > As it happens, this is due to the fact that E is a type, while E() is > > not. There's an optimization in the isinstance() machinery that simply > > checks to see if D().__class__ is a subtype of E. That's where your > > experiment fails. > > > > I'm not sure whether this behavior should be considered correct or not. > > > >Doesn't seems to be just related to isinstance implementation, That may be, but it's due to code that's entirely independent. isinstance() does what it does independently of the behavior you describe below, which is a product of how descriptors behave in new and old-style classes. The behavior you describe below is a natural consequence of the documented behavior of descriptors, while the above behavior is a performance optimization in isinstance(). They aren't related. >furthermore, it >is very surprising that old-style and new style classes behave exactly the >opposite way. > >In [2]: class a(object) : >...: __class__ = 0 >...: >...: > >In [3]: a.__class__ >Out[3]: > >In [4]: a().__class__ >Out[4]: 0 > >In [7]: class a : >...: __class__ = 0 >...: >...: > >In [8]: a.__class__ >Out[8]: 0 > >In [9]: a().__class__ >Out[9]: ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
At 02:49 PM 6/27/2006 +0200, Martin v. Löwis wrote: >Phillip J. Eby wrote: > >> It seems that the __class__ is only accessed in some cases, but not > >> always, leading to what I think is a semantic inconsistency. > > > > It's not inconsistent - isinstance() checks __class__ in *addition* to > > type() in order to allow proxying tricks like lying about your > > __class__. It therefore returns true if either your real type *or* your > > __class__ matches, and as you can see, the real type is checked first. > >This is not the original rationale, though: the check for a __class__ >attribute on non-instance objects was introduced in r13520, to support >ExtensionClasses. I never fully understood ExtensionClasses, but I >believe they were not based on proxying tricks. Instead, they were >an early version of new-style classes. Okay, well I recall discussion on zope-dev regarding making sure that isinstance() would support __class__ for security proxies as well. However, I do not recall from the discussion whether isinstance() already did this as an effect of the above, so it's possible that the discussion was regarding an existing behavior. In any event, ExtensionClasses are obsolete or becoming so, but security proxies are an ongoing need, and lying about __class__ is used by other projects besides Zope. I'm not aware of anything besides ExtensionClass that lies about __bases__, however, which might explain why that aspect of the behavior isn't as well fleshed out. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] doc for new restricted execution design for Python
(1) Is it impossible for an interpreter to switch between trusted and untrusted modes? This is probably a reasonable restriction, but worth calling out loudly in the docs. (2) For the APIs returning an int, it wasn't clear what that int would be, other than NULL => interpreter is trusted. I'm not sure that NULL is even always the best answer when the interpreter is trusted. For example, if I called PyXXX_AllowFile, I want to know whether the file is now allowed; I don't really care that it is allowed because the interpreter is trusted anyhow. (3) Should PyXXX_Trusted have a variant that takes group/type/string, meaning "Am I allowed to do *this*?", rather than having to special-case the "You can do anything" case? (4) For capped resources, there needs to be a way to tell what that cap is, and how much is left. (Logically, this provides "how much is already used", which is already a frequently requested feature for memory.) One use of untrusted interpreters is to stop runaway processes. For example, it might always be OK to add 4M memory, so long as it has been at least 10 seconds since the last request. This requires the controller to know what the current setting is. Caps and current usage should also be available (though read-only) from python; it is quite sensible to spill some cache when getting too close to your memory limit. (5) I think file creation/writing should be capped rather than binary; it is reasonable to say "You can create a single temp file up to 4K" or "You can create files, but not more than 20Meg total". (6) Given your expectation of one interpreter per web page, interpreters will have to be very lightweight. This might be the real answer to the CPU limiting -- just run each restricted interpreter as its own "thread" (possibly not an OS-level thread), and let the scheduler switch it out. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
On 6/27/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Mitch Chapman [1] tripped over the fact that relative imports don't like main
> modules because __name__ doesn't contain any package hierarchy information.
>
> It occurred to me that a slight modification to PEP 338 might solve the
> problem fairly cleanly: instead of simply setting __name__ to '__main__' for a
> module in a package, the -m switch could prepend the package name so that
> relative imports can work correctly.
>
> Inside the module, the test for "am I the main module" would need to be
> "__name__.endswith('__main__')" instead of "__name__ == '__main__'", but other
> than that, there should be very little impact.
>
> By sticking the main module into sys.modules under two different names (once
> with the package prefix and once without), runpy could even take care of
> ensuring that the following invariant held:
>
>import __main__
>by_name = __import__(__main__.__name__)
>assert by_name is __main__
>
> The behaviour of top level modules would be unaffected, since they won't have
> a package prefix.
>
> Cheers,
> Nick.
>
> [1] http://www.python.org/sf/1510172
Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you
don't want to mess with it.
Such a main module ought to use an *absolute* import to reach into the
rest of the package.
However, I'm fine with setting *another* variable to the full package
name so someone who *really* wants to do relative imports here knows
the package name.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On 6/26/06, K.S.Sreeram <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > I think we need a PEP for const/static/only/cached/precomputed or > > whatever people like to call it. > > fredrik's got a micro pep at http://online.effbot.org > > > Once we have (say) static, I think making the case expressions static > > by default would still cover all useful cases, and would allow us to > > diagnose duplicate cases reliably (which the if/elif chain semantics > > don't allow IIUC). > > Making case expressions default static would be very surprising to users > because of the restrictions placed by static. For instance 'case in a', > will not support containers which have a custom __contains__ method. It > will also not support containers like lists, and sets because they are > mutable. IMHO this doesn't feel very pythonic. > > Instead if we redefine the goal of the switch statement to be 'ease of > expression' rather than 'optimization', then it can just be used as a > concise alternative to if-elif chains, and we can make 'case in a' work > with all containers where a regular 'in' statement works *AND* still > give the possibility of fast lookup when the programmer wants, using > explicit static. > > I feel programmer expressivity is more important, and default static > case expressions looks like premature optimization. You've just placed yourself in School Ia (see the updated PEP). I respectfully disagree. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On 6/27/06, Robin Bryce <[EMAIL PROTECTED]> wrote: > > PEP 3103, When to Freeze the Dispatch Dict/Option 1 > > 2 things resonated with me for Raymond's proposal and the follow up: > > - It seemed agnostic to almost all of the independently contentious issues. Except for the need to use named constants. > - "is defined tightly enough to allow room for growth and elaboration over > time" [Raymond]. In particular it left room for > const/static/only/cached/etc to come along later. > > I think its worth acknowledging this in the PEP. Search for Raymond's name. It's there. > Is nothing better than something in this case ? I don't know. > > > I think we need a PEP for const/static/only/cached/precomputed or > > whatever people like to call it. > > > > Once we have (say) static, I think making the case expressions static > > by default would still cover all useful cases, and would allow us to > > diagnose duplicate cases reliably (which the if/elif chain semantics > > don't allow IIUC). > > If the expectation is that static/const will evolve as a sibling pep, > does this not make Raymond's suggestion any more appealing, even a > little ? No, then School Ia becomes more appealing. Raymond's proposal is unpythonic by not allowing expressions. > Is it unacceptable - or impractical - to break the addition of switch > to python in two (minor version separated) steps ? But what's the point? We have until Python 3000 anyway. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
On 27 Jun 2006, at 13:03, Nick Coghlan wrote:
> ...
> It occurred to me that a slight modification to PEP 338 might solve
> the
> problem fairly cleanly: instead of simply setting __name__ to
> '__main__' for a
> module in a package, the -m switch could prepend the package name
> so that
> relative imports can work correctly.
>
> Inside the module, the test for "am I the main module" would need
> to be
> "__name__.endswith('__main__')" instead of "__name__ ==
> '__main__'", but other
> than that, there should be very little impact.
Hum... other than effecting more or less every runnable python module
around it should be very little impact. That sounds like quite a bit
of impact to me!
Nicko
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On 6/27/06, Nick Coghlan <[EMAIL PROTECTED]> wrote:
> Guido van Rossum wrote:
> > I've written a new PEP, summarizing (my reaction to) the recent
> > discussion on adding a switch statement. While I have my preferences,
> > I'm trying to do various alternatives justice in the descriptions. The
> > PEP also introduces some standard terminology that may be helpful in
> > future discussions. I'm putting this in the Py3k series to gives us
> > extra time to decide; it's too important to rush it.
> >
> > http://www.python.org/dev/peps/pep-3103/
>
> A generally nice summary, but as one of the advocates of Option 2 when it
> comes to freezing the jump table, I'd like to see it given some better press
> :)
Sure. Feel free to edit the PEP directly if you want.
> > Feedback (also about misrepresentation of alternatives I don't favor)
> > is most welcome, either to me directly or as a followup to this post.
>
> My preferred variant of Option 2 (calculation of the jump table on first use)
> disallows function locals in the switch cases just like Option 3. The
> rationale is that the locals can't be expected to remain the same across
> different invocations of the function, so caching an expression that depends
> on them is just as nonsensical for Option 2 as it is for Option 3 (and hence
> should trigger a Syntax Error either way).
OK, but the explanation of Option 2 becomes more cumbersome then:
instead of "first time executed" it now is "first time executed and
you cannot use any locals (but you can use locals if you're executing
globally, and you can use locals of outer functions) (oh, and whether
locals in a class are okay is anybody's guess)."
> Given that variant, my reasons for preferring Option 2 over Option 3 are:
> - the semantics are the same at module, class and function level
No they're not. At the global level, this is okay bit at the function
level it's not:
C = 1
switch x:
case C: print 42
Unless I misunderstand you and you want to disallow locals at the
global level too, in which case I see this okay at the function level
but not at the global level:
switch x:
case re.IGNORECASE: print 42
So I don't see how this is really true.
> - the order of execution roughly matches the order of the source code
Only roughly though. One can still create obfuscated examples.
> - it does not cause any surprises when switches are inside conditional logic
>
> As an example of the latter kind of surprise, consider this:
>
>def surprise(x):
> do_switch = False
> if do_switch:
> switch x:
> case sys.stderr.write("Not reachable!\n"):
> pass
>
> Option 2 won't print anything, since the switch statement is never executed,
> so the jump table is never built. Option 3 (def-time calculation of the jump
> table), however, will print "Not reachable!" to stderr when the function is
> defined.
That's a pretty crooked example if you ask me. I think we all agree
that side effects of case expressions is one way how we can deduce the
compiler's behind-the-scenes tricks (even School Ib is okay with
this). So I don't accept this as proof that Option 2 is better.
> Now consider this small change, where the behaviour of Option 3 is not only
> surprising but outright undefined:
>
>def surprise(x):
> if 0:
> switch x:
> case sys.stderr.write("Not reachable!\n"):
> pass
>
> The optimiser is allowed to throw away the contents of an if 0: block. This
> makes no difference for Option 2 (since it never executed the case expression
> in the first place), but what happens under Option 3? Is "Not reachable!"
> written to stderr or not?
This is a good question. I think both behaviors are acceptable. Again,
the problem is with the side-effect-full case expression, not with
Option 3.
> When it comes to the question of "where do we store the result?" for the
> first-execution calculation of the jump table, my proposal is "a hidden cell
> in the current namespace".
Um, what do you mean by the current namespace? You can't mean the
locals of the function containing the switch. There aren't always
outer functions so I must conclude you mean the module globals. But
I've never seen those referred to as "the current namespace".
> The first time the switch statement is executed, the cell object is empty, so
> the jump table creation code is executed and the result stored in the cell. On
> subsequent executions of the switch statement, the jump table is retrieved
> directly from the cell.
OK.
> For functions, the cell objects for any switch tables would be created
> internally by the function object constructor based on the attributes of the
> code object. So the cells would be created anew each time the function
> definition is executed. These would be saved on the function object and
> inserted into the local namespace under the appropriate names before the code
> is executed (this is roughly the same thing that i
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote:
(1) Is it impossible for an interpreter to switch between trusted anduntrusted modes? This is probably a reasonable restriction, but worthcalling out loudly in the docs.Yes, you should not change the state once the interpreter is used for execution.
(2) For the APIs returning an int, it wasn't clear what that intwould be, other than NULL => interpreter is trusted.
Doesn't matter. I should probably change it to a say "a false value" instead of NULL.
I'm not sure that NULL is even always the best answer when theinterpreter is trusted. For example, if I called PyXXX_AllowFile, Iwant to know whether the file is now allowed; I don't really care thatit is allowed because the interpreter is trusted anyhow.
It's a question of whether you want that interpretation or want to make sure you never call the restriction setters on trusted interpreters. Anyone else have a preference like Jim?
(3) Should PyXXX_Trusted have a variant that takes group/type/string,meaning "Am I allowed to do *this*?", rather than having tospecial-case the "You can do anything" case?
The PyXXX_Trusted() case is meant as a blanket trusted/untrusted test. If you want more fine-grained, use the other checking functions (e.g., PyXXX_ExtendedCheckValue(), etc.). As I mention in the docs, if you want a "am I allowed to do this, if not I want to do something else", wrap the checking functions in another function and check that function's return value::
int check_for_value(group, type, string) { PyXXX_ExtendedCheckValue(group, type, string, 0); return 1; }
(4) For capped resources, there needs to be a way to tell what thatcap is, and how much is left. (Logically, this provides "how much isalready used", which is already a frequently requested feature for
memory.)Fair enough.One use of untrusted interpreters is to stop runaway processes. For
example, it might always be OK to add 4M memory, so long as it hasbeen at least 10 seconds since the last request. This requires thecontroller to know what the current setting is.Caps and current usage should also be available (though read-only)
from python; it is quite sensible to spill some cache when getting tooclose to your memory limit.Yeah, being able to read your restrictions seems reasonable to do from an untrusted interpreter.
(5) I think file creation/writing should be capped rather thanbinary; it is reasonable to say "You can create a single temp file up
to 4K" or "You can create files, but not more than 20Meg total".That has been suggested before. Anyone else like this idea?
(6) Given your expectation of one interpreter per web page,interpreters will have to be very lightweight. This might be the realanswer to the CPU limiting -- just run each restricted interpreter asits own "thread" (possibly not an OS-level thread), and let the
scheduler switch it out.That's another possibility; having the OS's threading capabilities run individual instances of the interpreter in its own thread instead of having Python manage all of the interpreters itself. I just don't know how feasible that is based on how Python is designed to be embedded and has so much global state.
Thanks for the feedback, Jim!-Brett
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote: > On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: > > (2) For the APIs returning an int, it wasn't clear what that int > > would be, other than NULL => interpreter is trusted. > Doesn't matter. I should probably change it to a say "a false value" > instead of NULL. But what if they succeed? Do they return -1, 1, the amount allocated, ... > > (3) Should PyXXX_Trusted have a variant that takes group/type/string, > > meaning "Am I allowed to do *this*?", rather than having to > > special-case the "You can do anything" case? > The PyXXX_Trusted() case is meant as a blanket trusted/untrusted test. If > you want more fine-grained, use the other checking functions (e.g., > PyXXX_ExtendedCheckValue(), etc.). You gave an example of a library that was generally useful even in restricted mode, but had one convenience function that shouldn't always be permitted. I imagine a function that is dangerous only because it takes a filename rather than an open stream; I want to wrap it in some sort of guard, but I would rather make a single "Can I do this?" query. Under the current API, I would need separate logic for "The interpreter is completely trusted" and "The interpreter is not trusted, but can do _this_". In practice, I'm betting that many extension modules will skip at least one of these steps. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
At 08:08 AM 6/27/2006 -0700, Guido van Rossum wrote: >Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you >don't want to mess with it. Actually, maybe we *do* want to, for this usage. Note that until Python 2.5, it was not possible to do "python -m nested.module", so this change merely prevents *existing* modules from being run this way -- when they could not have been before! So, such modules would require a minor change to run under -m. Is this actually a problem, or is it a new feature? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote:> On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote:> > (2) For the APIs returning an int, it wasn't clear what that int > > would be, other than NULL => interpreter is trusted.> Doesn't matter. I should probably change it to a say "a false value"> instead of NULL.But what if they succeed? Do they return -1, 1, the amount allocated, ... It can be specified as 1 or whatever. I just planned on a true value. > > (3) Should PyXXX_Trusted have a variant that takes group/type/string,> > meaning "Am I allowed to do *this*?", rather than having to> > special-case the "You can do anything" case? > The PyXXX_Trusted() case is meant as a blanket trusted/untrusted test. If> you want more fine-grained, use the other checking functions (e.g.,> PyXXX_ExtendedCheckValue(), etc.).You gave an example of a library that was generally useful even in restricted mode, but had one convenience function that shouldn'talways be permitted.I imagine a function that is dangerous only because it takes afilename rather than an open stream; I want to wrap it in some sort of guard, but I would rather make a single "Can I do this?" query.Well, for the filename operation it should be protected by the file object protection anyway, but ignoring this fact, I get your point. Under the current API, I would need separate logic for "Theinterpreter is completely trusted" and "The interpreter is not trusted, but can do _this_". In practice, I'm betting that manyextension modules will skip at least one of these steps.My worry with this is that by providing checking functions that just return true or false that people will rely on those too much and have logic errors in their check and let security holes develop. That is why the checking functions as they stand now are macros that do the error return for you. If people *really* need this I can add check functions and rename the current check functions to be more like "require" functions. I just want to hear other people say they will need this that often to warrant the risk of supporting possible security leaks from coding mistakes. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Is Lib/test/crashers/recursive_call.py really a crasher?
If you look at that crasher, you will notice that recursion depth is set to 1 << 30 before any code is run. If you remove that setting high setting and go with the default then the test doesn't crash and raises the appropriate RuntimeError. Setting the recursion depth to such a high number will crash the interpreter even when the proper recursion checks are in place. This doesn't seem like a legit crasher to me if it requires an insane recursion depth that would crash almost any C program that had recursion in it. Anyone have any objections if I call foul on the test and remove it without any changes to Python?-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
Phillip J. Eby wrote: > Actually, maybe we *do* want to, for this usage. > > Note that until Python 2.5, it was not possible to do "python -m > nested.module", so this change merely prevents *existing* modules from > being run this way -- when they could not have been before! > > So, such modules would require a minor change to run under -m. Is > this > actually a problem, or is it a new feature? This is where I wonder why the "def __main__()" PEP was rejected in the first place. It would have solved this problem as well. -- Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
On 6/27/06, Nicko van Someren <[EMAIL PROTECTED]> wrote:
> Hum... other than effecting more or less every runnable python module
> around it should be very little impact. That sounds like quite a bit
> of impact to me!
Going from "__name__ == '__main__'" to "__name__.endswith('__main__')"
can be handled by a search-and-replace function, so, yes, the impact
is indeed minimal. It's not like you have to manually recode every
usage.
Collin Winter
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
On 6/27/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 08:08 AM 6/27/2006 -0700, Guido van Rossum wrote: > >Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you > >don't want to mess with it. > > Actually, maybe we *do* want to, for this usage. > > Note that until Python 2.5, it was not possible to do "python -m > nested.module", so this change merely prevents *existing* modules from > being run this way -- when they could not have been before! > > So, such modules would require a minor change to run under -m. Is this > actually a problem, or is it a new feature? It's a feature with a problem, that's what it is. :-) My main concern is that people will feel the requirement to change all existing main programs to use the endswith() idiom whether they need it or not; there's a powerful meme that says you should be future-compatible and who knows when your script will end up as part of a package. So we'd see proliferation of the new idiom way beyond necessity, which would be a shame. I'd rather turn the argument around: if you had a "main" script that used your package before 2.5, the script would be required to use absolute import to access the package anyway. Presumably the script would be copied to somewhere on $PATH and the package would be copied somewhere on $PYTHONPATH (site-packages most likely) and the script would invoke the package via its full name. The new -m feature adds the possibility that exactly the same main script may now also be copied (with the rest of the package) onto $PYTHONPATH, without also copying it to $PATH, and it can be invoked using -m. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
On Tue, Jun 27, 2006, Phillip J. Eby wrote: > At 08:08 AM 6/27/2006 -0700, Guido van Rossum wrote: >> >>Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you >>don't want to mess with it. > > Actually, maybe we *do* want to, for this usage. > > Note that until Python 2.5, it was not possible to do "python -m > nested.module", so this change merely prevents *existing* modules from > being run this way -- when they could not have been before! > > So, such modules would require a minor change to run under -m. Is this > actually a problem, or is it a new feature? Well, yes, considering that cd'ing to the module's dir and doing "python module.py" will now fail. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "I saw `cout' being shifted "Hello world" times to the left and stopped right there." --Steve Gonedes ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is Lib/test/crashers/recursive_call.py really a crasher?
On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote: If you look at that crasher, you will notice that recursion depth is set to 1 << 30 before any code is run. If you remove that setting high setting and go with the default then the test doesn't crash and raises the appropriate RuntimeError. Setting the recursion depth to such a high number will crash the interpreter even when the proper recursion checks are in place. This doesn't seem like a legit crasher to me if it requires an insane recursion depth that would crash almost any C program that had recursion in it. Anyone have any objections if I call foul on the test and remove it without any changes to Python?Well, it's a valid crasher. It crashes Python to recurse too much. The recursion limit was added to CPython to prevent the crash from happening too easily, but that limit is just an implementation detail (and furthermore, the actual limit is just guessed.) It's not like a real solution is impossible, it's just very complex. Much like, say, restricted execution :-)-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote: > My worry with this is that by providing checking functions that just return > true or false that people will rely on those too much and have logic errors > in their check and let security holes develop. That is why the checking > functions as they stand now are macros that do the error return for you. Using a macro that returns an Error is OK. (Well, from this perspective; it might be a problem for reference leaks.) I just want a single call that does my erroring out, instead of two separate calls depending on whether the interpreter is trusted. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is Lib/test/crashers/recursive_call.py really a crasher?
On 6/27/06, Thomas Wouters <[EMAIL PROTECTED]> wrote: On 6/27/06, Brett Cannon < [EMAIL PROTECTED]> wrote: If you look at that crasher, you will notice that recursion depth is set to 1 << 30 before any code is run. If you remove that setting high setting and go with the default then the test doesn't crash and raises the appropriate RuntimeError. Setting the recursion depth to such a high number will crash the interpreter even when the proper recursion checks are in place. This doesn't seem like a legit crasher to me if it requires an insane recursion depth that would crash almost any C program that had recursion in it. Anyone have any objections if I call foul on the test and remove it without any changes to Python?Well, it's a valid crasher. It crashes Python to recurse too much. The recursion limit was added to CPython to prevent the crash from happening too easily, but that limit is just an implementation detail (and furthermore, the actual limit is just guessed.) It's not like a real solution is impossible, it's just very complex. Much like, say, restricted execution :-)OK, let me rephrase: I don't feel like fixing this if the proper thing happens when the default recursion depth is in place. There are a ton of other recursion issues if you set the recursion depth to 1,073,741,824. One could try to make the interpreter non-recursive or stackless, but I leave that to people who are smarter than me. =) And so, with that view, I don't see the test as something that needs special attention that is brought by being in crashers since I suspect that one will sit there forever. =)-BrettHi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
> Given that variant, my reasons for preferring Option 2 over Option 3 are:
> - the semantics are the same at module, class and function level
> - the order of execution roughly matches the order of the source code
> - it does not cause any surprises when switches are inside conditional logic
>
> As an example of the latter kind of surprise, consider this:
>
>def surprise(x):
> do_switch = False
> if do_switch:
> switch x:
> case sys.stderr.write("Not reachable!\n"):
> pass
>
> Option 2 won't print anything, since the switch statement is never executed,
> so the jump table is never built. Option 3 (def-time calculation of the jump
> table), however, will print "Not reachable!" to stderr when the function is
> defined.
Good points on order of define vs order of execution surprises.
WARNING: probable over generalization below or really usefull idea
depending on your point of view. ;)
I use dict base dispatching in a number of my programs and like it with
the exception I need to first define all the code in functions (or use
lambda) even if they are only one line. So it results in a three step
process, define functions, define dict, and then call it. And I need
to make sure all the function calls use the same calling signature. In
some cases I'm passing variables that one function doesn't need because
it is needed in one of the other cases.
So modeling the switch after dictionary dispatching more directly where
the switch is explicitly defined first and then used later might be good
both because it offers reuse in the current scope and it can easily be
used in code that currently uses dict style dispatching.
switch name:
1:
...
TWO:
...
'a', 'b', 'c':
...
in range(5,10):
...
else:
...
for choice in data:
do choice in name:# best calling form I can think of.
I think this avoids most of the define time/order and optimization
issues as well as the issues I have with dict base dispatching so I
thought it might be worth a mention. There may still be some advantage
to evaluating the case expressions early, but I think it might not be
needed as much in this form so they could be evaluated at switch
definition time, which is the order the code is written.
The main arguments against this form might be that it approaches macro
and named blocks capabilities a bit too closely, but those may also be
arguments for it as this may more directly fulfill the reasons some want
named blocks and or macros.
To use a named switch in such a way just call the desired case
explicitly ...
switch responses:
'make_red':
...
'make_blue':
...
do 'make_red' in responses:
...
do 'make_red' in responses: # again
...
do 'make_blue' in responses:
...
So it offers local reuse of code in a more direct way than a switch
statement does and more closely matches that which is current practice
with dictionary dispatching.
Cheers,
Ron
Just a thought,
Ron
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is Lib/test/crashers/recursive_call.py really a crasher?
"Brett Cannon" <[EMAIL PROTECTED]> writes: > If you look at that crasher, you will notice that recursion depth is set > to 1 << 30 before any code is run. If you remove that setting high > setting and go with the default then the test doesn't crash and raises the > appropriate RuntimeError. > > Setting the recursion depth to such a high number will crash the > interpreter even when the proper recursion checks are in place. This > doesn't seem like a legit crasher to me if it requires an insane recursion > depth that would crash almost any C program that had recursion in it. > > Anyone have any objections if I call foul on the test and remove it > without any changes to Python? Yes, it's still a way to crash Python :-) (in fact, a problem vaguely like this that made a complete test run segfault on 64-bit platforms was fixed in PyPy recently). More seriously, the recursion limit approach is IMHO something of a hack, as the amount of bytes of C stack in between increments is rather variable (try seeing how high you have to set the recursion limit to when the recursion invovles list.sort() compared to when it doesn't). I don't have a fantastic idea for fixing this, but I quite like having some kind of reminder of it. Cheers, mwh -- ZAPHOD: Who are you? ROOSTA: A friend. ZAPHOD: Oh yeah? Anyone's friend in particular, or just generally well-disposed to people? -- HHGttG, Episode 7 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On 6/27/06, Ron Adam <[EMAIL PROTECTED]> wrote: > I use dict base dispatching in a number of my programs and like it with > the exception I need to first define all the code in functions (or use > lambda) even if they are only one line. So it results in a three step > process, define functions, define dict, and then call it. And I need > to make sure all the function calls use the same calling signature. In > some cases I'm passing variables that one function doesn't need because > it is needed in one of the other cases. > > So modeling the switch after dictionary dispatching more directly where > the switch is explicitly defined first and then used later might be good > both because it offers reuse in the current scope and it can easily be > used in code that currently uses dict style dispatching. > > switch name: >1: > ... >TWO: > ... >'a', 'b', 'c': > ... >in range(5,10): > ... >else: > ... > > for choice in data: >do choice in name:# best calling form I can think of. It looks like your proposal is to change switch into a command that defines a function of one parameter. Instead of the "do in " call you could just call the switch -- no new syntax needed. Your example above would be for choice in data: name(choice) # 'name' is the switch's name However, early on in the switch discussion it was agreed that switch, like if/elif, should not create a new scope; it should just be a control flow statement sharing the surrounding scope. The switch as function definition would require the use of globals. Also, it would make sense if a switch could be a method instead of a function. I realize that by proposing a new invocation syntax (do ... in ...) you might have intended some other kind of interaction between the switch and the surrounding scope. but exactly what you're proposing isn't very clear from your examples, since you don't have any example code in the case suites, just "...". -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Misleading error message from PyObject_GenericSetAttr
Does anyone here have time to look at this? On 6/26/06, Alexander Belopolsky <[EMAIL PROTECTED]> wrote: > On 6/19/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > On 6/14/06, Alexander Belopolsky <[EMAIL PROTECTED]> wrote: > > > ... It would be better to change the message > > > to "'Foo' object has only read-only attributes (assign to .bar)" as in > > > the case tp_setattro == tp_setattr == NULL in PyObject_SetAttr . > > > > I agree. Can you submit a patch to SF please? > > > Please see: > > https://sourceforge.net/tracker/index.php?func=detail&aid=1512942&group_id=5470&atid=305470 > > I've tested the patch by setting tp_setattr to 0 in Xxo_Type. With the patch: > > >>> import xx > >>> x = xx.new() > >>> x.a = 2 > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'xxmodule.Xxo' object has only read-only attributes > (assign to .a) > >>> del x.a > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'xxmodule.Xxo' object has only read-only attributes (del .a) > > Note that this log reveals a small inaccuracy in xxmodule.c : the > module name is "xx," but Xxo type name is "xxmodule.Xxo." Should I > submit a patch fixing that? > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote:> My worry with this is that by providing checking functions that just return> true or false that people will rely on those too much and have logic errors > in their check and let security holes develop. That is why the checking> functions as they stand now are macros that do the error return for you.Using a macro that returns an Error is OK. (Well, from this perspective; it might be a problem for reference leaks.)Shouldn't be as long as you put the call right after variable declarations and you don't do an PyObject creation at variable declaration time. I just want a single call that does my erroring out, instead of twoseparate calls depending on whether the interpreter is trusted. Oh, you won't! You have the set call before you even start using the interpreter to define your restrictions; that has a return value to flag that you are trying to set restrictions on a trusted interpreter, and thus are trying to do somethign that just won't work. Then you have the check functions that run in *any* interpreter. If you happen to be running in a trusted interpreter, then they do nothing; basically a NOOP and allow execution to continue. But if you are running an untrusted interpreter, the check is performed. Does that make sense? In running code within an interpreter there is no trusted/untrusted distinction when it comes to using checking functions. The distinction only exists outside the interpreter before you begin using it. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is Lib/test/crashers/recursive_call.py really a crasher?
On 6/27/06, Michael Hudson <[EMAIL PROTECTED]> wrote: "Brett Cannon" <[EMAIL PROTECTED]> writes:> If you look at that crasher, you will notice that recursion depth is set> to 1 << 30 before any code is run. If you remove that setting high > setting and go with the default then the test doesn't crash and raises the> appropriate RuntimeError.>> Setting the recursion depth to such a high number will crash the> interpreter even when the proper recursion checks are in place. This > doesn't seem like a legit crasher to me if it requires an insane recursion> depth that would crash almost any C program that had recursion in it.>> Anyone have any objections if I call foul on the test and remove it > without any changes to Python?Yes, it's still a way to crash Python :-) (in fact, a problem vaguelylike this that made a complete test run segfault on 64-bit platformswas fixed in PyPy recently). More seriously, the recursion limit approach is IMHO something of ahack, as the amount of bytes of C stack in between increments israther variable (try seeing how high you have to set the recursionlimit to when the recursion invovles list.sort() compared to when itdoesn't). I don't have a fantastic idea for fixing this, but I quitelike having some kind of reminder of it.OK, with you and Thomas both wanting to keep it I will let it be. I just won't worry about fixing it myself during my interpreter hardening crusade. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] School IIb?
> On compilation, freeze any cases that meet the School-II conditions > and have a trustworthy __hash__ method into a dictionary. As long as the semantics are based on if-elif, you have to support if(optimizable) elif (has a side effect) elif (optimizable) elif (not optimizable) elif (optimizable) elif (has a side effect) elif (optimizable) where the four "optimizable" cases are actually in four separate dictionaries. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
Brett Cannon wrote: > On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: >> ... >> Caps and current usage should also be available (though read-only) >> from python; it is quite sensible to spill some cache when getting too >> close to your memory limit. > > Yeah, being able to read your restrictions seems reasonable to do from an > untrusted interpreter. Certainly in some cases I'd like to run a Python program that claims it "plays nice" without its being able to see that it is in jail. Otherwise I can't escalate my trust of the code based on old behavior (it might be nice only when the jailer is around). So, reading your restrictions is a capability I'd like to be able to control. -- Scott David Daniels [EMAIL PROTECTED] ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Scott David Daniels <[EMAIL PROTECTED]> wrote: Brett Cannon wrote:> On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote:>> ...>> Caps and current usage should also be available (though read-only) >> from python; it is quite sensible to spill some cache when getting too>> close to your memory limit.>> Yeah, being able to read your restrictions seems reasonable to do from an> untrusted interpreter. Certainly in some cases I'd like to run a Python program that claims it"plays nice" without its being able to see that it is in jail. OtherwiseI can't escalate my trust of the code based on old behavior (it might be nice only when the jailer is around). So, reading your restrictions isa capability I'd like to be able to control.Sounds reasonable.-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote: > On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: > > > On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote: > Shouldn't be as long as you put the call right after variable declarations > and you don't do an PyObject creation at variable declaration time. When PEPping this, please add that restriction to the Extension Module Crippling section. > > I just want a single call that does my erroring out, instead of two > > separate calls depending on whether the interpreter is trusted. > Oh, you won't! You have the set call before you even start using the > interpreter to define your restrictions; that has a return value to flag > that you are trying to set restrictions on a trusted interpreter, and thus > are trying to do somethign that just won't work. Then you have the check > functions that run in *any* interpreter. This is what I was missing -- the bit about who uses which part of the API. Is the following correct: Py_XXXCheck* and Py_XXXExtendedCheck* are called by C extension modules. They error out of the current function if the action would not be allowed. (In the special case of of a fully trusted function, the happen to compile themselves out.) There may be some Py_XXXInfo functions added to find out what the limits are, particularly for python code. Py_XXXTrusted() should really be renamed Py_XXXCheckTrusted(). Crippled extension modules should really use Py_XXXCheck*, but PyXXXCheckTrusted is a quick way to get all-or-nothing. No other PyXXX functions should ever be (directly) called by any loadable module, not even by C extension modules; they are called only by an embedding program. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal to eliminate PySet_Fini
Setobject code allocates several internal objects on the heap that are cleaned up by the PySet_Fini function. This is a fine design choice, but it often makes debugging applications with embedded python more difficult. I propose to eliminate the need for PySet_Fini as follows: 1. Make dummy and emptyfrozenset static objects similar to Py_None 2. Eliminate the free sets reuse scheme. The second proposal is probably more controversial, but is there any real benefit from that scheme when pymalloc is enabled? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote:> On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote:>> > On 6/27/06, Brett Cannon < [EMAIL PROTECTED]> wrote:> Shouldn't be as long as you put the call right after variable declarations> and you don't do an PyObject creation at variable declaration time. When PEPping this, please add that restriction to the Extension ModuleCrippling section.Sure. > > I just want a single call that does my erroring out, instead of two> > separate calls depending on whether the interpreter is trusted.> Oh, you won't! You have the set call before you even start using the > interpreter to define your restrictions; that has a return value to flag> that you are trying to set restrictions on a trusted interpreter, and thus> are trying to do somethign that just won't work. Then you have the check > functions that run in *any* interpreter.This is what I was missing -- the bit about who uses which part of the API.Is the following correct:Py_XXXCheck* and Py_XXXExtendedCheck* are called by C extension modules. They error out of the current function if the action wouldnot be allowed. (In the special case of of a fully trusted function,the happen to compile themselves out.)They don't compile themselves out unless you didn't compile the functionality in at all, but yes, that's right. There may be some Py_XXXInfo functions added to find out what thelimits are, particularly for python code. Yep. Once the C API is settled equivalents at the Python level will be dealt with. Py_XXXTrusted() should really be renamed Py_XXXCheckTrusted().Crippled extension modules should really use Py_XXXCheck*, butPyXXXCheckTrusted is a quick way to get all-or-nothing.Rename seems reasonable. And yes, that is the right idea of usage. No other PyXXX functions should ever be (directly) called by anyloadable module, not even by C extension modules; they are called only by an embedding program.Yep.I think I will try to add a paragraph at the top using pseudocode, showing typical usage.-Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Do we need a bug triage day?
Do we need a sort of mini bug-day to look at the outstanding bugs and note ones that absolutely need to be fixed before 2.5final? Or has someone already done this? --amk ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Guido van Rossum wrote:
> On 6/27/06, Ron Adam <[EMAIL PROTECTED]> wrote:
>> I use dict base dispatching in a number of my programs and like it with
>> the exception I need to first define all the code in functions (or use
>> lambda) even if they are only one line. So it results in a three step
>> process, define functions, define dict, and then call it. And I need
>> to make sure all the function calls use the same calling signature. In
>> some cases I'm passing variables that one function doesn't need because
>> it is needed in one of the other cases.
>>
>> So modeling the switch after dictionary dispatching more directly where
>> the switch is explicitly defined first and then used later might be good
>> both because it offers reuse in the current scope and it can easily be
>> used in code that currently uses dict style dispatching.
>>
>> switch name:
>>1:
>> ...
>>TWO:
>> ...
>>'a', 'b', 'c':
>> ...
>>in range(5,10):
>> ...
>>else:
>> ...
>>
>> for choice in data:
>>do choice in name:# best calling form I can think of.
>
> It looks like your proposal is to change switch into a command that
> defines a function of one parameter. Instead of the "do
> in " call you could just call the switch -- no new syntax
> needed. Your example above would be
>
> for choice in data:
>name(choice) # 'name' is the switch's name
I thought of using a function call so it would be more like using a
generator, but also ruled it out because it does create a new scope and
I think closures may complicate it or it would require also passing all
the names needed for each case which would get old quick if it is
required every time. One of the things I want to be able to avoid in
dict based dispatching for cases with only one or two lines of code.
So my intent was that it use the local scope and not use the function
call signature which implies a new scope to the reader and a returned
value, thus the 'do choice in name' calling form. No returned value is
needed because it has full access to local name space.
for example you wouldn't write...
return if x: 42 else: 84
but would instead...
if x:
y = 42
else:
y = 84
return y
The 'do' is used in the same context an 'if' is used.
switch a:
True: y=42
else: y=84
do x in a:
return y
> However, early on in the switch discussion it was agreed that switch,
> like if/elif, should not create a new scope; it should just be a
> control flow statement sharing the surrounding scope. The switch as
> function definition would require the use of globals.
>
> Also, it would make sense if a switch could be a method instead of a
> function.
There's no reason why it couldn't be put "in" a method. If the switch
uses the surrounding name space you have that flexibility. I'm not sure
if the select definition could be put in the body of a class and have
the do's in a method. That would be like having an if in the body of the
class and the else to it in a method, so I would think it wouldn't be
allowed. So they both would need to be in the same name space and the
select will always need to be defined before the 'do' is executed.
> I realize that by proposing a new invocation syntax (do ... in ...)
> you might have intended some other kind of interaction between the
> switch and the surrounding scope. but exactly what you're proposing
> isn't very clear from your examples, since you don't have any example
> code in the case suites, just "...".
What was intended probably would be more closely related to constructing
a switch with BASICS gosub command.
one: # in basic these do not have their own scope
print 'one'
return # return from subroutine not function here
two:
print 'two'
return
three:
print 'three'
return
data = ('one', 'two', 'three')
for choice in data:
if choice == 'one': gosub one
elif choice == 'two': gosub two
elif choice == 'three': gosub three
Which would be better expressed as..
switch choices:
'one': print 'one'
'two': print 'two'
'three': print 'three'
for choice in ('one', 'two', 'three'):
do choice in choices
Each case label expression would be evaluated when the switch block is
executed, ie... in order it appears in the program, but the code for
each case would be skipped until a (do choice in choices) line. Each
switch case block would not fall through but return to the next line
after the 'do' line by default.
The whole thing could be put in a separate function or method if it's
desired to get the single function call form you suggested along with a
separate name space.
def switcher(choice):
switcher roo:
1: a = 42
42: a = 1
else: raise ValueError
do choice i
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
On 6/27/06, Ron Adam <[EMAIL PROTECTED]> wrote:
> Guido van Rossum wrote:
> > It looks like your proposal is to change switch into a command that
> > defines a function of one parameter. Instead of the "do
> > in " call you could just call the switch -- no new syntax
> > needed. Your example above would be
> >
> > for choice in data:
> >name(choice) # 'name' is the switch's name
>
> I thought of using a function call so it would be more like using a
> generator, but also ruled it out because it does create a new scope and
> I think closures may complicate it or it would require also passing all
> the names needed for each case which would get old quick if it is
> required every time. One of the things I want to be able to avoid in
> dict based dispatching for cases with only one or two lines of code.
>
> So my intent was that it use the local scope and not use the function
> call signature which implies a new scope to the reader and a returned
> value, thus the 'do choice in name' calling form. No returned value is
> needed because it has full access to local name space.
>
> for example you wouldn't write...
>
> return if x: 42 else: 84
>
> but would instead...
>
> if x:
> y = 42
> else:
> y = 84
> return y
>
>
> The 'do' is used in the same context an 'if' is used.
>
> switch a:
> True: y=42
> else: y=84
>
> do x in a:
> return y
Ah, I see.
> > However, early on in the switch discussion it was agreed that switch,
> > like if/elif, should not create a new scope; it should just be a
> > control flow statement sharing the surrounding scope. The switch as
> > function definition would require the use of globals.
> >
> > Also, it would make sense if a switch could be a method instead of a
> > function.
>
> There's no reason why it couldn't be put "in" a method. If the switch
> uses the surrounding name space you have that flexibility. I'm not sure
> if the select definition could be put in the body of a class and have
> the do's in a method. That would be like having an if in the body of the
> class and the else to it in a method, so I would think it wouldn't be
> allowed. So they both would need to be in the same name space and the
> select will always need to be defined before the 'do' is executed.
>
> > I realize that by proposing a new invocation syntax (do ... in ...)
> > you might have intended some other kind of interaction between the
> > switch and the surrounding scope. but exactly what you're proposing
> > isn't very clear from your examples, since you don't have any example
> > code in the case suites, just "...".
>
> What was intended probably would be more closely related to constructing
> a switch with BASICS gosub command.
I understand now.
But I have a question: if I write
for i in range(10):
switch S:
case i: print 42
(i.e. the switch is *inside* the for loop) does the switch get defined
10 times (with 10 different case values!) or not?
> one: # in basic these do not have their own scope
> print 'one'
> return # return from subroutine not function here
>
> two:
> print 'two'
> return
>
> three:
> print 'three'
> return
>
> data = ('one', 'two', 'three')
> for choice in data:
> if choice == 'one': gosub one
> elif choice == 'two': gosub two
> elif choice == 'three': gosub three
>
>
> Which would be better expressed as..
>
>
> switch choices:
> 'one': print 'one'
> 'two': print 'two'
> 'three': print 'three'
>
> for choice in ('one', 'two', 'three'):
> do choice in choices
I'm not sure I like the idea of using BASIC as a way to explain Python
functionality... :-)
> Each case label expression would be evaluated when the switch block is
> executed, ie... in order it appears in the program, but the code for
> each case would be skipped until a (do choice in choices) line. Each
> switch case block would not fall through but return to the next line
> after the 'do' line by default.
>
> The whole thing could be put in a separate function or method if it's
> desired to get the single function call form you suggested along with a
> separate name space.
>
> def switcher(choice):
> switcher roo:
>1: a = 42
>42: a = 1
>else: raise ValueError
>
> do choice in switcher:
> return a
>
> switcher(1) -> 42
> switcher(42) -> 1
> switcher(100) -> raises exception
I'm still unclear on when you propose the case expressions to be
evaluated. Each time the "switch" statement is encountered? That would
be the most natural given the rest of your explanation. But then a
switch inside a function that references globally defined constants
would be re-evalulated each time the function is called; much of the
discussion here is focused on trying to reduce the number of times the
switch cases are evaluated t
Re: [Python-Dev] Switch statement - handling errors
On 6/26/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > I like Python's rules to be simple, and I > prefer to occasionally close off a potential optimization path in the > sake of simplicity. (Almost) everyone agrees that the case expressions SHOULD be run-time constants. The disagreements are largely over what to do when this gets violated. Bad Case Option (1) -- Accept everything Readability is everything. Switch/case tells you that every branch is using similar predicates on the same variable. If that variable or predicate can't be easily optimized, then so what -- it is still better to read. (Largely School Ia) Bad Case Option (2) -- Accept very little - Enforce good case expressions. (Raymond's proposal) This does the right thing when it works, but it is initially very restricted -- and a crippled case statement may be worse than none at all. Bad Case Option (3) -- Raise Exceptions --- Flag bugs early. The semantics require non-overlapping hashable constants, so raise an exception if this gets violated. This does the right thing, but catching all the violations in a timely manner is hard. Freezing at first use is too late for a good exception, but any earlier has surprising restrictions. There is no good way to realize that a "constant" has changed after the freeze. Bad Case Option (4) -- Ignore problems -- This is for optimization; go ahead and ignore any problems you can. Maybe that branch will never be taken... Ironically, this is also largely school I, since it matches the if semantics. Bad Case Option (5) -- ad hoc mixture - Pick an arbitrary set of rules, and follow it. Guido is currently leaning towards this, with the rules being "freeze at definition", raise for unhashable, ignore later changes, undecided on overlapping ranges. The disadvantage is that people can "cheat" with non-constant expressions. Sometimes, this will work out great. Sometimes it will lead to nasty non-localized bugs. We have to explain exactly which cheats are allowed, and that explanation could get byzantine. Bad Case Option (6) -- Undefined Undefined behavior. We don't yet know which strategy (or mix of strategies) is best. So don't lock ourselves (and Jython, and PyPy, and IronPython, and ShedSkin, and ...) into the wrong strategy. The down side is that people may start to count on the actual behavior anyhow; then (in practice) we might just have Bad Case Option (5) without documentation. -jJ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
> But what's the point? We have until Python 3000 anyway. Ah, my mistake. In my enthusiasm, I foolishly got the time frames of peps 3103 & 275 mixed up. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Split switch statement
One of the big problems here seems to be that an optimized switch statement requires some code to be evaluated fewer times than the rest of the switch statement, and there isn't any good way to make that happen with a single statement. Thus, I propose two statements: a 'dispatcher' statement and a 'switch' statement. The dispatcher statement defines the available cases, and generates a dispatcher object, and the switch statement specifies code to be run for each case. - #Sample 1: Primary dispatcher syntax dispatcher myEvents on e: on e: case None: Idle on getattr(e, 'type', None): from globalEvents: * case 42: myEvent # or whatever the user event code is #EOF A dispatcher statement contains a sequence of 'on' blocks. Each 'on' block specifies an expression and a set of cases. The expression is stored as a lambda inside the dispatcher which is applied whenever the switch is run. Inside a 'on' block, there are two kinds of statements. 'case' evaluates its expression immediately, and associates it with a label; 'from' imports tests and labels from another dispatcher. If the result of any case expression is unhashable, an exception is raised. -- #Sample 2: Shorthand dispatcher syntax dispatcher globalEvents: case pygame.KEYDOWN: KEYDOWN case pygame.KEYUP: KEYUP ... #EOF Because dispatching on the switched value directly is so common, any 'from' or 'case' statements outside an 'on' block are considered to be applied to be inside an "on " block. The name for the switched value can be omitted if it's not needed. -- #Sample 3: Using a dispatcher while True: ... switch events on pygame.event.poll(): case KEYUP, KEYDOWN: ... case myEvent: ... case Idle: ... else: ... #EOF Internally, each switch statement has some unique identifier. Each dispatcher object maintains a list of the switch statements it has previously serviced. If this switch statement is new to this dispatcher, the dispatcher verifies that it might generate all of the cases that are specified in the switch. Otherwise, an exception is raised. If the test passed (or was skipped due to previous experience), each of the 'on' expressions in the dispatcher is executed (in order) and their results are checked against the stored values. If no case (from the switch, not the dispatcher) matches, the switch's 'else' block is executed, if present. If more than one case (from the switch) matches, an exception is raised. Otherwise, the code from associated case block is executed. -- Eric Sumner PS. Yes, I know that's not how pygame handles idle events; it makes a better sample this way. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Guido van Rossum wrote: >> What was intended probably would be more closely related to constructing >> a switch with BASICS gosub command. > > I understand now. > > But I have a question: if I write > > for i in range(10): >switch S: > case i: print 42 > > (i.e. the switch is *inside* the for loop) does the switch get defined > 10 times (with 10 different case values!) or not? In this instance the switch would be redefined 10 times. The ending switch would be: switch S: case 10: print 42 The static keyword could be used with this form as well to force define time evaluation. (see last example.) > I'm not sure I like the idea of using BASIC as a way to explain Python > functionality... :-) Yes, I agree! :-) Fortunately, once (and if) it's defined (what ever it turns out to be) Python examples can be used to explain Python. ;-) >> Each case label expression would be evaluated when the switch block is >> executed, ie... in order it appears in the program, but the code for >> each case would be skipped until a (do choice in choices) line. Each >> switch case block would not fall through but return to the next line >> after the 'do' line by default. >> >> The whole thing could be put in a separate function or method if it's >> desired to get the single function call form you suggested along with a >> separate name space. >> >> def switcher(choice): >> switcher roo: >>1: a = 42 >>42: a = 1 >>else: raise ValueError >> >> do choice in switcher: >> return a >> >> switcher(1) -> 42 >> switcher(42) -> 1 >> switcher(100) -> raises exception > > I'm still unclear on when you propose the case expressions to be > evaluated. Each time the "switch" statement is encountered? That would > be the most natural given the rest of your explanation. But then a > switch inside a function that references globally defined constants > would be re-evalulated each time the function is called; much of the > discussion here is focused on trying to reduce the number of times the > switch cases are evaluated to once per program invocation or once per > function *definition*. Each time the 'switch' statement is encountered for the above. Allowing static to be used with this form could be an option to force function define time evaluations of cases. ONE = 1 FOURTYTWO = 42 def switcher(choice): static switcher roo: # evaluate cases at function def time. ONE: a = 42 FOURTYTWO: a = 1 else: raise ValueError do choice in switcher: return a switcher(1) -> 42 switcher(42) -> 1 switcher(100) -> raises exception That would give you both call time and def time evaluations for cases with clear behavior for both (I think). Also since the switch has a name, it might be possible to examine their values with dir(switch_suite_name). That might help in debugging and explaining the behavior in different situations. The 'case' keyword could be left off in this form because the switch body is always a suite. I find it more readable without it in the case of simple literals or named values, and more readable with it for more complex expressions. I don't think I can clarify this further without getting over my head. I probably am a bit already. I'm in the know enough to get myself in trouble (at times) group. ;-) Ron ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Ron Adam wrote: > In this instance the switch would be redefined 10 times. The ending > switch would be: > > switch S: >case 10: print 42 Silly mistake correction... :) switch S: case 9: print 42 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement - handling errors
On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: > On 6/26/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > I like Python's rules to be simple, and I > > prefer to occasionally close off a potential optimization path in the > > sake of simplicity. > > (Almost) everyone agrees that the case expressions SHOULD be run-time > constants. The disagreements are largely over what to do when this > gets violated. Thanks for the elaboration. I'm not sure how to respond except to correct your representation of my position: > Bad Case Option (1) -- Accept everything > > > Readability is everything. Switch/case tells you that every branch is > using similar predicates on the same variable. If that variable or > predicate can't be easily optimized, then so what -- it is still > better to read. (Largely School Ia) > > > Bad Case Option (2) -- Accept very little > - > > Enforce good case expressions. (Raymond's proposal) This does the > right thing when it works, but it is initially very restricted -- and > a crippled case statement may be worse than none at all. > > > Bad Case Option (3) -- Raise Exceptions > --- > > Flag bugs early. The semantics require non-overlapping hashable > constants, so raise an exception if this gets violated. This does the > right thing, but catching all the violations in a timely manner is > hard. > > Freezing at first use is too late for a good exception, but any > earlier has surprising restrictions. > > There is no good way to realize that a "constant" has changed after the > freeze. > > > Bad Case Option (4) -- Ignore problems > -- > > This is for optimization; go ahead and ignore any problems you can. > Maybe that branch will never be taken... Ironically, this is also > largely school I, since it matches the if semantics. > > > Bad Case Option (5) -- ad hoc mixture > - > > Pick an arbitrary set of rules, and follow it. > > Guido is currently leaning towards this, with the rules being "freeze > at definition", raise for unhashable, ignore later changes, undecided > on overlapping ranges. Actually I'm all for flagging overlapping changes as errors when the dict is frozen. > The disadvantage is that people can "cheat" with non-constant > expressions. Sometimes, this will work out great. Sometimes it will > lead to nasty non-localized bugs. We have to explain exactly which > cheats are allowed, and that explanation could get byzantine. Actually I would simply explain that all cheats are frowned upon, just like all side effects in case expressions. > > > Bad Case Option (6) -- Undefined > > > Undefined behavior. We don't yet know which strategy (or mix of > strategies) is best. So don't lock ourselves (and Jython, and PyPy, > and IronPython, and ShedSkin, and ...) into the wrong strategy. > > The down side is that people may start to count on the actual behavior > anyhow; then (in practice) we might just have Bad Case Option (5) > without documentation. > > -jJ > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
Martin v. Löwis wrote: > Again, I believe this is all included for ExtensionClasses: it looks > for __class__ on the object if the type check fails, so that an > ExtensionClass could be actually a class derived from the C type. Now that we have had new-style classes for quite a while, is there still a need to support ExtensionClasses? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
Guido van Rossum wrote: > Bad idea IMO. The __name__ == "__main__" rule is so ingrained, you > don't want to mess with it. It would only make a difference for main modules inside packages. Wouldn't that be fairly rare? The vast majority of existing __name__ == "__main__" uses ought to be unaffected. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
Giovanni Bajo wrote: > This is where I wonder why the "def __main__()" PEP was rejected in the > first place. It would have solved this problem as well. Could this be reconsidered for Py3k? -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantic of isinstance
At 12:02 PM 6/28/2006 +1200, Greg Ewing wrote: >Martin v. Löwis wrote: > > > Again, I believe this is all included for ExtensionClasses: it looks > > for __class__ on the object if the type check fails, so that an > > ExtensionClass could be actually a class derived from the C type. > >Now that we have had new-style classes for quite a >while, is there still a need to support ExtensionClasses? That's the wrong question. The right question is, "is there a need to support isinstance() for proxy objects?" and the answer is yes. As far as I know, nobody has proposed to change this behavior of isinstance(), nor even suggested a reason for doing so. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 328 and PEP 338, redux
On 6/27/06, Greg Ewing <[EMAIL PROTECTED]> wrote: > Giovanni Bajo wrote: > > > This is where I wonder why the "def __main__()" PEP was rejected in the > > first place. It would have solved this problem as well. > > Could this be reconsidered for Py3k? You have a point. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
> The plan is to allow pure Python code to be embedded into web pages like > JavaScript. I am not going for the applet approach like Java. Java support is now just a plug-in. Should be easy to make a Python plug-in system that works the same way. If only we had a GUI... :-) Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal to eliminate PySet_Fini
Alexander Belopolsky wrote: > Setobject code allocates several internal objects on the heap that are > cleaned up by the PySet_Fini function. This is a fine design choice, > but it often makes debugging applications with embedded python more > difficult. given that CPython has about a dozen Fini functions, what exactly is it that makes PySet_Fini so problematic ? $ more Python/pythonrun.c ... PyMethod_Fini(); PyFrame_Fini(); PyCFunction_Fini(); PyTuple_Fini(); PyList_Fini(); PySet_Fini(); PyString_Fini(); PyInt_Fini(); PyFloat_Fini(); _PyUnicode_Fini(); ... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement - handling errors
On Tue, 27 Jun 2006, Jim Jewett wrote: > (Almost) everyone agrees that the case expressions SHOULD be run-time > constants. The disagreements are largely over what to do when this > gets violated. I like your summary and understood most of it (options 1, 2, 3, 5, 6). The only part i didn't understand was this: > Bad Case Option (4) -- Ignore problems > -- > > This is for optimization; go ahead and ignore any problems you can. > Maybe that branch will never be taken... Ironically, this is also > largely school I, since it matches the if semantics. Could you elaborate on what this means? Does "ignore any problems" mean "even if a case value changes, pretend it didn't change"? But that wouldn't match the 'if' semantics, so i'm not sure what you had in mind. > Bad Case Option (6) -- Undefined > [...] > The down side is that people may start to count on the actual behavior > anyhow; then (in practice) we might just have Bad Case Option (5) > without documentation. I agree with this last paragraph. Option 6 seems the most risky of all. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] doc for new restricted execution design for Python
On 6/27/06, Brett Cannon <[EMAIL PROTECTED]> wrote:
>
> > (5) I think file creation/writing should be capped rather than
> > binary; it is reasonable to say "You can create a single temp file up
> > to 4K" or "You can create files, but not more than 20Meg total".
>
> That has been suggested before. Anyone else like this idea?
What would this code do:
MAX = 4
for i in xrange(10):
fp = open(str(i), 'w+')
fp.write(' ' * (MAX // 4))
fp.close()
if i % 2:
os.unlink(str(i))
How many times should this execute, 4 or 8? What about if there is no
if i % 2 and the file is unlinked at the end of each loop? Should
that loop 10 times without error? What would happen if we used the
same file name? What would happen if we did something like:
fp = open(str(i), 'w+')
MAX = 4
for i in xrange(1):
fp.seek(0)
fp.write(' ' * (MAX // 4))
Should this succeed?
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Switch statement - handling errors
On 6/27/06, Jim Jewett <[EMAIL PROTECTED]> wrote: > Bad Case Option (5) -- ad hoc mixture > - > > Pick an arbitrary set of rules, and follow it. > > Guido is currently leaning towards this, with the rules being "freeze > at definition", raise for unhashable, ignore later changes, undecided > on overlapping ranges. > > The disadvantage is that people can "cheat" with non-constant > expressions. Sometimes, this will work out great. Sometimes it will > lead to nasty non-localized bugs. We have to explain exactly which > cheats are allowed, and that explanation could get byzantine. A solution that is often offered in situations like this: Pychecker (or something like it) can do a much more thorough check. It should be easy for Pychecker to keep track of the constancy of variables. IMO my proposal has no cheats. It has well-defined semantics. Maybe not the semantics you'd like to see, but without ESP built into the compiler that's impossible. If you stick to the simple rule "constants only" then you won't see any semantics surprises. Just stay away from anything where you're not sure whether it's really a constant, and you'll be fine. If on the other hand you want to explore the boundaries of the semantics, we'll give you one simple rule, and you can verify that that rule is indeed all there is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3103: A Switch/Case Statement
Guido van Rossum wrote: > On 6/27/06, Ron Adam <[EMAIL PROTECTED]> wrote: > >>So modeling the switch after dictionary dispatching more directly where >>the switch is explicitly defined first and then used later might be good >>both because it offers reuse in the current scope and it can easily be >>used in code that currently uses dict style dispatching. >> >>switch name: >> 1: >> ... >> TWO: >> ... >> 'a', 'b', 'c': >> ... >> in range(5,10): >> ... >> else: >> ... >> >>for choice in data: >> do choice in name:# best calling form I can think of. > > > It looks like your proposal is to change switch into a command that > defines a function of one parameter. Instead of the "do > in " call you could just call the switch -- no new syntax > needed. Your example above would be > > for choice in data: > name(choice) # 'name' is the switch's name This parallels some of my thinking -- that we ought to somehow make the dict-building aspect of the switch statement explicit (which is better than implicit, as we all have been taught.) My version of this is to add to Python the notion of a simple old-fashioned subroutine - that is, a function with no arguments and no additional scope, which can be referred to by name. For example: def MyFunc( x ): sub case_1: ... sub case_2: ... sub case_3: ... # A direct call to the subroutine: do case_1 # An indirect call y = case_2 do y # A dispatch through a dict d = dict( a=case_1, b=case_2, c_case_3 ) do d[ 'a' ] The 'sub' keyword defines a subroutine. A subroutine is simply a block of bytecode with a return op at the end. When a subroutine is invoked, control passes to the indented code within the 'sub' clause, and continues to the end of the block - there is no 'fall through' to the next block. When the subroutine is complete, a return instruction is exected, and control transfers back to the original location. Because subroutines do not define a new scope, they can freely modify the variables of the scope in which they are defined, just like the code in an 'if' or 'else' block. One ambiguity here is what happens if you attempt to call a subroutine from outside of the code block in which it is defined. The easiest solution is to declare that this is an error - in other words, if the current execution scope is different than the scope in which the subroutine is defined, an exception is thrown. A second possibility is to store a reference to the defining scope as part of the subroutine definition. So when you take a reference to 'case_1', you are actually referring to a closure of the enclosing scope and the subroutine address. This approach has a number of advantages that I can see: -- Completely eliminates the problems of when to freeze the dict, because the dict is 'frozen' explicitly (or not at all, if desired.) -- Completely eliminates the question of whether to support ranges in the switch cases. The programmer is free to invent whatever type of dispatch mechanism they wish. For example, instead of using a dict, they could use an array of subroutines, or a spanning tree / BSP tree to represent contiguous ranges of options. -- Allows for development of dispatch methods beyond the switch model - for example, the dictionary could be computed, transformed and manipulated by user code before used for dispatch. -- Allows for experimentation with other flow of control forms. The primary disadvantage of this form is that the case values and the associated code blocks are no longer co-located, which reduces some of the expressive power of the switch. Note that if you don't want to define a new keyword, an alternate syntax would be 'def name:' with no argument braces, indicating that this is not a function but a procedure. -- Talin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
