Re: [Python-Dev] Re: anonymous blocks
On Thu, 28 Apr 2005, Greg Ewing wrote: > If such an explanation can't be found, I strongly suspect > that this doesn't correspond to a cohesive enough concept > to be made into a built-in language feature. If you can't > give a short, understandable explanation of it, then it's > probably a bad idea. In general, i agree with the sentiment of this -- though it's also okay if there is a way to break the concept down into concepts that *are* simple enough to have short, understandable explanations. -- ?!ng ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: switch statement
> "Guido" == Guido van Rossum <[EMAIL PROTECTED]> writes: Guido> You mean like this? if x > 0: ...normal case... elif y > 0: abnormal case... else: ...edge case... The salient example! If it's no accident that those conditions are mutually exclusive and exhaustive, doesn't that code require at least a comment saying so, and maybe even an assertion to that effect? Where you can use a switch, it gives both, and throws in economy in both source and object code as a bonus. Not a compelling argument---your example shows switches are not universally applicable---but it's a pretty good deal. Guido> You have guts to call that bad style! :-) Exaggeration in defense of elegance is no vice. -- School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp University of TsukubaTennodai 1-1-1 Tsukuba 305-8573 JAPAN Ask not how you can "do" free software business; ask what your business can "do for" free software. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Greg Ewing wrote: Guido van Rossum wrote: And surely you exaggerate. How about this then: The with-statement is similar to the for-loop. Until you've learned about the differences in detail, the only time you should write a with-statement is when the documentation for the function you are calling says you should. I think perhaps I'm not expressing myself very well. What I'm after is a high-level explanation that actually tells people something useful, and *doesn't* cop out by just saying "you're not experienced enough to understand this yet". this makes sense to me, also because a new control statement will not be usually as hidden as metaclasses and some other possibly obscure corners can be. OTOH I have the impression that the new toy is too shiny to have a lucid discussion whether it could have sharp edges or produce dizziness for the unexperienced. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Anonymous blocks: Thunks or iterators?
Elegant as the idea behind PEP 340 is, I can't shake the feeling that it's an abuse of generators. It seems to go to a lot of trouble and complication so you can write a generator and pretend it's a function taking a block argument. I'd like to reconsider a thunk implementation. It would be a lot simpler, doing just what is required without any jiggery pokery with exceptions and break/continue/return statements. It would be easy to explain what it does and why it's useful. Are there any objective reasons to prefer a generator implementation over a thunk implementation? If for-loops had been implemented with thunks, we might never have created generators. But generators have turned out to be more powerful, because you can have more than one of them on the go at once. Is there a use for that capability here? I can think of one possible use. Suppose you want to acquire multiple resources; one way would be to nest block-statements, like block opening(file1) as f: block opening(file2) as g: ... If you have a lot of resources to acquire, the nesting could get very deep. But with the generator implementation, you could do something like block iterzip(opening(file1), opening(file2)) as f, g: ... provided iterzip were modified to broadcast __next__ arguments to its elements appropriately. You couldn't do this sort of thing with a thunk implementation. On the other hand, a thunk implementation has the potential to easily handle multiple block arguments, if a suitable syntax could ever be devised. It's hard to see how that could be done in a general way with the generator implementation. [BTW, I've just discovered we're not the only people with numbered things called PEPs. I typed "PEP 340" into Google and got "PEP 340: Prevention and Care of Athletic Injuries"!] Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Integrating PEP 310 with PEP 340
Guido van Rossum wrote: PEP 310 forms the basis for a block construct that I _do_ like. The question then becomes whether or not generators can be used to write useful PEP 310 style block managers (I think they can, in a style very similar to that of the looping block construct from PEP 340). I've read through your example, and I'm not clear why you think this is better. It's a much more complex API with less power. What's your use case? Why should 'block' be disallowed from looping? TOOWTDI or do you have something better? I'm no longer clear on why I thought what I suggested would be better either. Can I use the 'it was late' excuse? :) Actually, the real reason is that I hadn't figured out what was really possible with PEP 340. The cases that I thought PEP 310 would handle better, I've since worked out how to do using the PEP 340 mechanism, and PEP 340 handles them _far_ more elegantly. With PEP 340, multi-stage constructs can be handled by using one generator as an argument to the block, and something else (such as a class or another generator) to maintain state between the blocks. The looping nature is a big win, because it lets execution of a contained block be prevented entirely. My favourite discovery is that PEP 340 can be used to write a switch statement like this: block switch(value) as sw: block sw.case(1): # Handle case 1 block sw.case(2): # Handle case 2 block sw.default(): # Handle default case Given the following definitions: class _switch(object): def __init__(self, switch_var): self.switch_var = switch_var self.run_default = True def case(self, case_value): self.run_default = False if self.switch_var == case_value: yield def default(self): if self.run_default: yield def switch(switch_var): yield _switch(switch_var) With the keyword-less syntax previously mentioned, such a 'custom structure' could look like: switch(value) as sw: sw.case(1): # Handle case 1 sw.case(2): # Handle case 2 sw.default(): # Handle default case (Actually doing a switch using blocks like this would be *insane* for performance reasons, but it is still rather cool that it is possible) With an appropriate utility block manager PEP 340 can also be used to abstract multi-stage operations. I haven't got a real use case for this as yet, but the potential is definitely there: def next_stage(itr): """Execute a single stage of a multi-stage block manager""" arg = None next_item = next(itr) while True: if next_item is StopIteration: raise StopIteration try: arg = yield next_item except: if not hasattr(itr, "__error__"): raise next_item = itr.__error__(sys.exc_info()[1]) else: next_item = next(itr, arg) def multi_stage(): """Code template accepting multiple suites""" # Pre stage 1 result_1 = yield # Post stage 1 yield StopIteration result_2 = 0 if result_1: # Pre stage 2 result_2 = yield # Post stage 2 yield StopIteration for i in range(result_2): # Pre stage 3 result_3 = yield # Post stage 3 yield StopIteration # Pre stage 4 result_4 = yield # Post stage 4 def use_multistage(): blk = multi_stage() block next_stage(blk): # Stage 1 continue val_1 block next_stage(blk): # Stage 2 continue val_2 block next_stage(blk): # Stage 3 continue val_3 block next_stage(blk): # Stage 4 continue val_4 Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
On 4/28/05, Steven Bethard <[EMAIL PROTECTED]> wrote: > however, the iterable object is notified whenever a 'continue', > 'break', or 'return' statement is executed inside the block-statement. This should read: however, the iterable object is notified whenever a 'continue', 'break' or 'return' statement is executed *or an exception is raised* inside the block-statement. Sorry! STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: switch statement
> Exaggeration in defense of elegance is no vice. Maybe not, but it still sounds like BS to me. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Greg Ewing wrote: Elegant as the idea behind PEP 340 is, I can't shake the feeling that it's an abuse of generators. It seems to go to a lot of trouble and complication so you can write a generator and pretend it's a function taking a block argument. I'd like to reconsider a thunk implementation. It would be a lot simpler, doing just what is required without any jiggery pokery with exceptions and break/continue/return statements. It would be easy to explain what it does and why it's useful. "Simple is better than Complex." Is there a thunk PEP? Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
On 4/28/05, Greg Ewing <[EMAIL PROTECTED]> wrote: > Guido van Rossum wrote: > > And surely you exaggerate. How about this then: > > > > The with-statement is similar to the for-loop. Until you've > > learned about the differences in detail, the only time you should > > write a with-statement is when the documentation for the function > > you are calling says you should. > > I think perhaps I'm not expressing myself very well. > What I'm after is a high-level explanation that actually > tells people something useful, and *doesn't* cop out by > just saying "you're not experienced enough to understand > this yet". How about: """ A block-statement is much like a for-loop, and is also used to iterate over the elements of an iterable object. In a block-statement however, the iterable object is notified whenever a 'continue', 'break', or 'return' statement is executed inside the block-statement. Most iterable objects do not need to be notified of such statement executions, so for most iteration over iterable objects, you should use a for-loop. Functions that return iterable objects that should be used in a block-statement will be documented as such. """ If you need more information, you could also include something like: """ When generator objects are used in a block-statement, they are guaranteed to be "exhausted" at the end of the block-statement. That is, any additional call to next() with the generator object will produce a StopIteration. """ STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Problem with embedded python - bug?
I have been having a few more discussions around about this, and I'm starting to think that this is a bug. My take is that, when I call Py_Finalize, the python thread should be shut down gracefully, closing the file and everything. Maybe I'm missing a call to something (?PyEval_FinalizeThreads?) but the docs seem to say that just PyFinalize should be called. The open file seems to be the issue, since if I remove all the references to the file I cannot get the program to crash. I can reproduce the same behavior on two different wxp systems, under python 2.4 and 2.4.1. Ugo -Original Message- From: Ugo Di Girolamo Sent: Tuesday, April 26, 2005 2:16 PM To: 'python-dev@python.org' Subject: Problem with embedded python I have the following code, that seems to make sense to me. However, it crashes about 1/3 of the times. My platform is Python 2.4.1 on WXP (I tried the release version from the msi and the debug version built by me, both downloaded today to have the latest version). The crash happens while the main thread is in Py_Finalize. I traced the crash to _Py_ForgetReference(op) in object.c at line 1847, where I have op->_ob_prev == NULL. What am I doing wrong? I'm definitely not too sure about the way I'm handling the GIL. Thanks in adv for any suggestion/ comment Cheers and ciao Ugo // TestPyThreads.py // #include #include "Python.h" int main() { PyEval_InitThreads(); Py_Initialize(); PyGILState_STATE main_restore_state = PyGILState_UNLOCKED; PyGILState_Release(main_restore_state); // start the thread { PyGILState_STATE state = PyGILState_Ensure(); int trash = PyRun_SimpleString( "import thread\n" "import time\n" "def foo():\n" " f = open('pippo.out', 'w', 0)\n" " i = 0;\n" " while 1:\n" "f.write('%d\\n'%i)\n" "time.sleep(0.01)\n" "i += 1\n" "t = thread.start_new_thread(foo, ())\n" ); PyGILState_Release(state); } // wait 300 ms Sleep(300); PyGILState_Ensure(); Py_Finalize(); return 0; } ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Brett C. wrote: Guido van Rossum wrote: Yet another alternative would be for the default behaviour to be to raise Exceptions, and continue with anything else, and have the third argument be "raise_exc=True" and set it to False to pass an exception in without raising it. You've lost me there. If you care about this, can you write it up in more detail (with code samples or whatever)? Or we can agree on a 2nd arg to __next__() (and a 3rd one to next()). Channeling Nick, I think he is saying that the raising argument should be made True by default and be named 'raise_exc'. Pretty close, although I'd say 'could' rather than 'should', as it was an idle thought, rather than something I actually consider a good idea. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
On 4/28/05, Greg Ewing <[EMAIL PROTECTED]> wrote: > Neil Schemenauer wrote: > > > The translation of a block-statement could become: > > > > itr = EXPR1 > > arg = None > > while True: > > try: > > VAR1 = next(itr, arg) > > except StopIteration: > > break > > try: > > arg = None > > BLOCK1 > > except Exception, exc: > > err = getattr(itr, '__error__', None) > > if err is None: > > raise exc > > err(exc) > > That can't be right. When __error__ is called, if the iterator > catches the exception and goes on to do another yield, the > yielded value needs to be assigned to VAR1 and the block > executed again. It looks like your version will ignore the > value from the second yield and only execute the block again > on the third yield. Could you do something like: itr = EXPR1 arg = None next_func = next while True: try: VAR1 = next_func(itr, arg) except StopIteration: break try: arg = None next_func = next BLOCK1 except Exception, arg: try: next_func = type(itr).__error__ except AttributeError: raise arg ? STeVe -- You can wordify anything if you just verb it. --- Bucky Katt, Get Fuzzy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: switch statement
On 4/28/05, Stephen J. Turnbull <[EMAIL PROTECTED]> wrote: > > "Guido" == Guido van Rossum <[EMAIL PROTECTED]> writes: > > Guido> You mean like this? > > if x > 0: > ...normal case... > elif y > 0: > abnormal case... > else: > ...edge case... > > The salient example! If it's no accident that those conditions are > mutually exclusive and exhaustive, doesn't that code require at least > a comment saying so, and maybe even an assertion to that effect? I usually do: if ...: return ... if ...: return ... assert ... return ... Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] noob question regarding the interpreter
Hello, I know this is a n00b question, so I apologize ahead of time. I've been taking a look at they python interpreter, trying to understand how it works on the compiled byte-codes. Looking through the sources of the 2.4.1 stable version, it looks like Python/ceval.c is the module that does the main dispatch. However, it looks like a switched interpreter. I just find this surprising because python seems to run pretty fast, and a switched interpreter is usually painfully slow. Is there work to change python into a direct-threaded or even JIT'ed interpreter? Has there been previous discussion on this topic? I'd greatly appreciate any pointers to discussions on this topic. Thus far my google-fu has not turned up fruitful hits. Thanks in advance for any help! -Jing ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] anonymous blocks as scope-collapse: detailed proposal
Based on Guido's opinion that caller and callee should both be marked, I have used keywords 'include' and 'chunk'. I therefore call them "Chunks" and "Includers". Examples are based on (1) The common case of a simple resource manager. e.g. http://mail.python.org/pipermail/python-dev/2005-April/052751.html (2) Robert Brewer's Object Relational Mapper http://mail.python.org/pipermail/python-dev/2005-April/052924.html which uses several communicating Chunks in the same Includer, and benefits from Includer inheritance. Note that several cooperating Chunks may use the same name (e.g. old_children) to refer to the same object, even though that object is never mentioned by the Includer. It is possible for the same code object to be both a Chunk and an Includer. Its own included sub-Chunks also share the top Includer's namespace. Chunks and Includers must both be written in pure python, because C frames cannot be easily manipulated. They can of course call or be called (as a unit) by extension modules. I have assumed that Chunks should not take arguments. While arguments are useful ("Which pattern should I match against on this inclusion?"), the same functionality *can* be had by binding a known name in the Includer. When that starts to get awkward, it is a sign that you should be using separate namespaces (and callbacks, or value objects). "self" and "cls" are just random names to a Chunk, though using them for any but the conventional meaning will be as foolhardy as it is in a method. Chunks are limited to statement context, as they do not return a value. Includers must provide a namespace. Therefore a single inclusion will turn the entire nearest enclosing namespace into an Includer. ? Should this be limited to nearest enclosing function or method? I can't think of a good use case for including directly from class definition or module toplevel, except registration. And even then, a metaclass might be better. Includers may only be used in a statement context, as the Chunks must be specified in a following suite. (It would be possible to skip the suite if all Chunk names are already bound, but I'm not sure that is a good habit to encourage -- so initially forbid it.) Chunks are defined without a (), in analogy to parentless classes. They are included (called) with a (), so that they can remain first class objects. Example Usage = def withfile(filename, mode='r'): """Close the file as soon we're done. This frees up file handles sooner. This is particularly important under Jython, or if you are using files in cyclic structures.""" openfile = open(filename, mode) try: include fileproc() # keyword 'include' prevents XXX_FAST optimization finally: openfile.close() chunk nullreader: # callee Chunk defined for reuse for line in openfile: pass withfile("testr.txt"): # Is this creation of a new block-starter a problem? fileproc=nullreader # Using an external Chunk object withfile("testw.txt", "w"): chunk fileproc: # Providing an "inline" Chunk openfile.write("Line 1") # If callers must be supported in expression context #fileproc=nullreader #withfile("tests.txt") # Resolve Chunk name from caller's default # binding, which in this case defaults back # to the current globals. # Is this just asking for trouble? class ORM(object): chunk nullchunk:# The extra processing is not always needed. pass begin=pre=post=end=nullchunk# Default to no extra processing def __set__(self, unit, value): include self.begin() if self.coerce: value = self.coerce(unit, value) oldvalue = unit._properties[self.key] if oldvalue != value: include self.pre() unit._properties[self.key] = value include self.post() include self.end() class TriggerORM(ORM): chunk pre: include super(self,TriggerORM).pre()# self was bound by __set__ old_children = self.children() # inject new variable chunk post: include super(self,TriggerORM).post() for child in self.children(): if child not in old_children: # will see pre's binding notify_somebody("New child %s" % child) As Robert Brewer said, > The above is quite ugly written with callbacks (due to > excessive argument passing), and is currently fragile > when overriding __set__ (due to duplicated code). How to Implement The Includer cannot know which variables a Chunk will use (or inject), so the namespace must remain a dictionary. This precludes use of the XXX_FAST bytecodes. But as Robert pointed out, avoiding another frame creation/destruction will compensate somewhat. Two new byt
Re: [Python-Dev] Re: anonymous blocks
Brett C. wrote: I'm surprisingly close to agreeing with you, actually. I've worked out that it isn't the looping that I object to, it's the inability to get out of the loop without exhausting the entire iterator. 'break' isn't' enough for you as laid out by the proposal? The raising of StopIteration, which is what 'break' does according to the standard, should be enough to stop the loop without exhausting things. Same way you stop a 'for' loop from executing entirely. The StopIteration exception effectively exhausted the generator, though. However, I've figured out how to deal with that, and my reservations about PEP 340 are basically gone. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.skystorm.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Greg Ewing <[EMAIL PROTECTED]> writes: > Are there any objective reasons to prefer a generator > implementation over a thunk implementation? I, too, would like to see an answer to this question. I'd like to see an answer in the PEP, too. Cheers, mwh -- All obscurity will buy you is time enough to contract venereal diseases. -- Tim Peters, python-dev ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
[Greg Ewing] > Elegant as the idea behind PEP 340 is, I can't shake > the feeling that it's an abuse of generators. It seems > to go to a lot of trouble and complication so you > can write a generator and pretend it's a function > taking a block argument. Maybe. You're not the first one saying this and I'm not saying "no" outright, but I'd like to defend the PEP. There are a number of separate ideas that all contribute to PEP 340. One is turning generators into more general coroutines: continue EXPR passes the expression to the iterator's next() method (renamed to __next__() to work around a compatibility issue and because it should have been called that in the first place), and in a generator this value can be received as the return value of yield. Incidentally this makes the generator *syntax* more similar to Ruby (even though Ruby uses thunks, and consequently uses return instead of continue to pass a value back). I'd like to have this even if I don't get the block statement. The second is a solution for generator resource cleanup. There are already two PEPs proposing a solution (288 and 325) so I have to assume this addresses real pain! The only new twist offered by PEP 340 is a unification of the next() API and the resource cleanup API: neither PEP 288 nor PEP 325 seems to specify rigorously what should happen if the generator executes another yield in response to a throw() or close() call (or whether that should even be allowed); PEP 340 takes the stance that it *is* allowed and should return a value from whatever call sent the exception. This feels "right", especially together with the previous feature: if yield can return a value as if it were a function call, it should also be allowed to raise an exception, and catch or propagate it with impunity. Even without a block-statement, these two changes make yield look a lot like invoking a thunk -- but it's more efficient, since calling yield doesn't create a frame. The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). You can't use a yield-based block for that (except in Ruby, which uses yield syntax with a thunk-based implementation). But I have to say that I almost see this as an advantage: I think I'd be slightly uncomfortable seeing a block and not knowing whether it will be executed in the normal control flow or later. Defining an explicit nested function for that purpose doesn't have this problem for me, because I already know that the 'def' keyword means its body is executed later. The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). But then an IMO important use case for the resource cleanup template pattern is lost. I routinely write code like this: def findSomething(self, key, default=None): self.lock.acquire() try: for item in self.elements: if item.matches(key): return item return default finally: self.lock.release() and I'd be bummed if I couldn't write this as def findSomething(self, key, default=None): block synchronized(self.lock): for item in self.elements: if item.matches(key): return item return default This particular example can be rewritten using a break: def findSomething(self, key, default=None): block synchronized(self.lock): for item in self.elements: if item.matches(key): break else: item = default return item but it looks forced and the transformation isn't always that easy; you'd be forced to rewrite your code in a single-return style which feels too restrictive. > I'd like to reconsider a thunk implementation. It > would be a lot simpler, doing just what is required > without any jiggery pokery with exceptions and > break/continue/return statements. It would be easy > to explain what it does and why it's useful. I don't know. In order to obtain the required local variable sharing between the thunk and the containing function I believe that every local variable used or set in the thunk would have to become a 'cell' (our mechanism for sharing variables between nested scopes). Cells slow down access somewhat compared to regular local variables. Perhaps not entirely coincidentally, the last example above (findSomething() rewritten to avoid a return inside the block) shows that, unlike for regular nested functions, we'll want variables *assigned to* by the thunk also to be sh
Re: [Python-Dev] Re: anonymous blocks
[Greg Ewing] > I think perhaps I'm not expressing myself very well. > What I'm after is a high-level explanation that actually > tells people something useful, and *doesn't* cop out by > just saying "you're not experienced enough to understand > this yet". > > If such an explanation can't be found, I strongly suspect > that this doesn't correspond to a cohesive enough concept > to be made into a built-in language feature. If you can't > give a short, understandable explanation of it, then it's > probably a bad idea. [Ping] > In general, i agree with the sentiment of this -- though it's > also okay if there is a way to break the concept down into > concepts that *are* simple enough to have short, understandable > explanations. I don't know. What exactly is the audience supposed to be of this high-level statement? It would be pretty darn impossible to explain even the for-statement to people who are new to programming, let alone generators. And yet explaining the block-statement *must* involve a reference to generators. I'm guessing most introductions to Python, even for experienced programmers, put generators off until the "advanced" section, because this is pretty wild if you're not used to a language that has something similar. (I wonder how you'd explain Python generators to an experienced Ruby programmer -- their mind has been manipulated to the point where they'd be unable to understand Python's yield no matter how hard they tried. :-) If I weren't limited to newbies (either to Python or to programming in general) but simply had to explain it to Python programmers pre-Python-2.5, I would probably start with a typical example of the try/finally idiom for acquiring and releasing a lock, then explain how for software engineering reasons you'd want to templatize that, and show the solution with a generator and block-statement. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 340 - possible new name for block-statement
How about, instead of trying to emphasize how different a block-statement is from a for-loop, we emphasize their similarity? A regular old loop over a sequence or iterable is written as: for VAR in EXPR: BLOCK A variation on this with somewhat different semantics swaps the keywords: in EXPR for VAR: BLOCK If you don't need the variable, you can leave the "for VAR" part out: in EXPR: BLOCK Too cute? :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] noob question regarding the interpreter
On Thu, Apr 28, 2005 at 05:47:18PM -0400, Jing Su wrote: > Is there work to change python into a direct-threaded or even JIT'ed > interpreter? People have experimented with making the ceval loop use direct threading. If I recall correctly, the resulting speedup was not significant. I suspect the reason is that most of Python's opcodes do a significant amount of work. There's probably more to be gained by moving to a register based VM. Also, I think direct threading is hard to do portably. If you are interested in JIT, take a look at Psyco. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 - possible new name for block-statement
On Thu, Apr 28, 2005 at 03:55:03PM -0700, Guido van Rossum wrote: > A variation on this with somewhat different semantics swaps the keywords: > > in EXPR for VAR: > BLOCK Looks weird to my eyes. On a related note, I was thinking about the extra cleanup 'block' provides. If the 'file' object would provide a suitable iterator, you could write: block open(filename) as line: ... and have the file closed at the end of the block. It does not read so well though. In a way, it seems to make more sense if 'block' called iter() on the expression and 'for' did not. block would guarantee to cleanup iterators that it created. 'for' does not but implictly creates them. Neil ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: PEP 340 - possible new name for block-statement
Guido van Rossum wrote: A variation on this with somewhat different semantics swaps the keywords: in EXPR for VAR: BLOCK If you don't need the variable, you can leave the "for VAR" part out: in EXPR: BLOCK Too cute? :-) I don't think it reads well. I would prefer something that would be understandable for a newbie's eyes, even if it fits more with common usage than with the real semantics behind it. For example a Boost-like keyword like: scoped EXPR as VAR: BLOCK scoped EXPR: BLOCK We may argue that it doesn't mean a lot, but at least if a newbie sees the following code, he would easily guess what it does: scoped synchronized(mutex): scoped opening(filename) as file: ... When compared with: in synchronized(mutex): in opening(filename) for file: ... As a C++ programmer, I still dream I could also do: scoped synchronized(mutex) scoped opening(filename) as file ... which would define a block until the end of the current block... Regards, Nicolas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] noob question regarding the interpreter
> However, it looks like a switched interpreter. I just > find this surprising because python seems to run pretty fast, and a switched > interpreter is usually painfully slow. This just proves how worthless a generalization that is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Guido van Rossum wrote: > I don't know. What exactly is the audience supposed to be of this > high-level statement? It would be pretty darn impossible to explain > even the for-statement to people who are new to programming, let alone > generators. And yet explaining the block-statement *must* involve a > reference to generators. I'm guessing most introductions to Python, > even for experienced programmers, put generators off until the > "advanced" section, because this is pretty wild if you're not used to > a language that has something similar. (I wonder how you'd explain > Python generators to an experienced Ruby programmer -- their mind has > been manipulated to the point where they'd be unable to understand > Python's yield no matter how hard they tried. :-) I think this concept can be explained clearly. I'd like to try explaining PEP 340 to someone new to Python but not new to programming. I'll use the term "block iterator" to refer to the new type of iterator. This is according to my limited understanding. "Good programmers move commonly used code into reusable functions. Sometimes, however, patterns arise in the structure of the functions rather than the actual sequence of statements. For example, many functions acquire a lock, execute some code specific to that function, and unconditionally release the lock. Repeating the locking code in every function that uses it is error prone and makes refactoring difficult. "Block statements provide a mechanism for encapsulating patterns of structure. Code inside the block statement runs under the control of an object called a block iterator. Simple block iterators execute code before and after the code inside the block statement. Block iterators also have the opportunity to execute the controlled code more than once (or not at all), catch exceptions, or receive data from the body of the block statement. "A convenient way to write block iterators is to write a generator. A generator looks a lot like a Python function, but instead of returning a value immediately, generators pause their execution at "yield" statements. When a generator is used as a block iterator, the yield statement tells the Python interpreter to suspend the block iterator, execute the block statement body, and resume the block iterator when the body has executed. "The Python interpreter behaves as follows when it encounters a block statement based on a generator. First, the interpreter instantiates the generator and begins executing it. The generator does setup work appropriate to the pattern it encapsulates, such as acquiring a lock, opening a file, starting a database transaction, or starting a loop. Then the generator yields execution to the body of the block statement using a yield statement. When the block statement body completes, raises an uncaught exception, or sends data back to the generator using a continue statement, the generator resumes. At this point, the generator can either clean up and stop or yield again, causing the block statement body to execute again. When the generator finishes, the interpreter leaves the block statement." Is it understandable so far? Shane ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Guido van Rossum wrote: The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). Or pass it on to another function. This is something we haven't considered -- what if one resource-acquision- generator (RAG?) wants to delegate to another RAG? With normal generators, one can always use the pattern for x in sub_generator(some_args): yield x But that clearly isn't going to work if the generators involved are RAGs, because the exceptions passed in are going to be raised at the point of the yield in the outer RAG, and the inner RAG isn't going to get finalized (assuming the for-loop doesn't participate in the finalization protocol). To get the finalization right, the inner generator needs to be invoked as a RAG, too: block sub_generator(some_args): yield But PEP 340 doesn't say what happens when the block contains a yield. A thunk implementation wouldn't have any problem with this, since the thunk can be passed down any number of levels before being called, and any exceptions raised in it will be propagated back up through all of them. The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Urg, you're right. Unless return is turned into an exception in that case. And then I suppose break and return (and yield?) will have to follow suit. I'm just trying to think how Smalltalk handles this, since it must have a similar problem, but I can't remember the details. every local variable used or set in the thunk would have to become a 'cell' . Cells slow down access somewhat compared to regular local variables. True, but is the difference all that great? It's just one more C-level indirection, isn't it? we'll want variables *assigned to* by the thunk also to be shared with the containing function, Agreed. We'd need to add a STORE_CELL bytecode or something for this. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Integrating PEP 310 with PEP 340
Nick Coghlan wrote: With an appropriate utility block manager I've just thought of another potential name for them: Block Utilization and Management Function (BUMF) :-) -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
Guido van Rossum wrote: Even without a block-statement, these two changes make yield look a lot like invoking a thunk -- but it's more efficient, since calling yield doesn't create a frame. I like PEP 340 a lot, probably as much or more than any thunk ideas I've seen. But I want to defend thunks here a little. It is possible to implement thunks without them creating their own frame. They can reuse the frame of the surrounding function. So a new frame does not need to be created when the thunk is called, and, much like with a yield statement, the frame is not taken down when the thunk completes running. The implementation just needs to take care to save and restore members of the frame that get clobbered when the thunk is running. Cells would of course not be required if the thunk does not create its own frame. The main advantage of thunks that I can see is that you can save the thunk for later, like a callback for a button widget (the thunk then becomes a closure). You can't use a yield-based block for that (except in Ruby, which uses yield syntax with a thunk-based implementation). But I have to say that I almost see this as an advantage: I think I'd be slightly uncomfortable seeing a block and not knowing whether it will be executed in the normal control flow or later. Defining an explicit nested function for that purpose doesn't have this problem for me, because I already know that the 'def' keyword means its body is executed later. I would also be uncomfortable if the thunk could be called at a later time. This can be disallowed by raising an exception if such an attempt is made. Such a restriction would not be completely arbitrary. One consequence of having the thunk borrow its surrounding function's frame is that it does not make much sense, implementationally speaking, to allow the thunk to be called at a later time (although I do realize that "it's difficult to implement" is not a good argument for anything). The other problem with thunks is that once we think of them as the anonymous functions they are, we're pretty much forced to say that a return statement in a thunk returns from the thunk rather than from the containing function. Doing it any other way would cause major weirdness when the thunk were to survive its containing function as a closure (perhaps continuations would help, but I'm not about to go there :-). If it is accepted that the thunk won't be callable at a later time, then I think it would seem normal that a return statement would return from the surrounding function. -Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 - possible new name for block-statement
Neil Schemenauer wrote: A variation on this with somewhat different semantics swaps the keywords: in EXPR for VAR: BLOCK Looks weird to my eyes. Probably makes perfect sense if you're Dutch, though. :-) -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 - possible new name for block-statement
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Guido van Rossum wrote: | How about, instead of trying to emphasize how different a | block-statement is from a for-loop, we emphasize their similarity? | | A regular old loop over a sequence or iterable is written as: | | for VAR in EXPR: | BLOCK | | A variation on this with somewhat different semantics swaps the keywords: | | in EXPR for VAR: | BLOCK | | If you don't need the variable, you can leave the "for VAR" part out: | | in EXPR: | BLOCK | | Too cute? :-) Far too close to the "for" loop, IMHO. I read that, I'd have to remind myself every time, "now, which one is it that can receive values passed back in: for ... in, or in ... for?" I'm definitely -1 on that one: too confusing. Another possibility just occurred to me. How about "using"? ~using EXPR as VAR: ~BLOCK Reads similarly to "with", but leaves the "with" keyword open for possible use later. Since it seems traditional for one to introduce oneself upon first posting to python-dev, my name is Robin Munn. Yes, my name is just one letter different from Robin Dunn's. It's not like I *intended* to cause confusion... :-) Anyway, I was introduced to Python a few years ago, around version 2.1 or so, and fell in love with the fact that I could read my own code six months later and understand it. I try to help out where I can, but I don't know the guts of the interpreter, so on python-dev I mostly lurk. Robin Munn -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCcbf16OLMk9ZJcBQRAuYpAJ4n24AgsX3SrW0g7jlWJM+HfzHXMwCfTbTq eJ2mLzg1uLZv09KDUemM+WU= =SXux -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Guido van Rossum wrote: I don't know. What exactly is the audience supposed to be of this high-level statement? It would be pretty darn impossible to explain even the for-statement to people who are new to programming, let alone generators. If the use of block-statements becomes common for certain tasks such as opening files, it seems to me that people are going to encounter their use around about the same time they encounter for-statements. We need *something* to tell these people to enable them to understand the code they're reading. Maybe it would be sufficient just to explain the meanings of those particular uses, and leave the full general explanation as an advanced topic. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Steven Bethard wrote: """ A block-statement is much like a for-loop, and is also used to iterate over the elements of an iterable object. No, no, no. Similarity to a for-loop is the *last* thing we want to emphasise, because the intended use is very different from the intended use of a for-loop. This is going to give people the wrong idea altogether. -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Anonymous blocks: Thunks or iterators?
(BTW, I'm trying to update the PEP with a discussion of thunks.) [Guido] > > The main advantage of thunks that I can see is that you can save the > > thunk for later, like a callback for a button widget (the thunk then > > becomes a closure). [Greg] > Or pass it on to another function. This is something we > haven't considered -- what if one resource-acquision- > generator (RAG?) wants to delegate to another RAG? > > With normal generators, one can always use the pattern > >for x in sub_generator(some_args): > yield x > > But that clearly isn't going to work if the generators > involved are RAGs, because the exceptions passed in > are going to be raised at the point of the yield in > the outer RAG, and the inner RAG isn't going to get > finalized (assuming the for-loop doesn't participate > in the finalization protocol). > > To get the finalization right, the inner generator > needs to be invoked as a RAG, too: > >block sub_generator(some_args): > yield > > But PEP 340 doesn't say what happens when the block > contains a yield. The same as when a for-loop contains a yield. The sub_generator is entirely unaware of this yield, since the local control flow doesn't actually leave the block (i.e., it's not like a break, continue or return statement). When the loop that was resumed by the yield calls next(), the block is resumed back after the yield. The generator finalization semantics guarantee (within the limitations of all finalization semantics) that the block will be resumed eventually. I'll add this to the PEP, too. I'd say that a yield in a thunk would be more troublesome: does it turn the thunk into a generator or the containing function? It would have to be the thunk, but then things get weird quickly (the caller of the thunk has to treat the result of the call as an iterator). > A thunk implementation wouldn't have any problem with > this, since the thunk can be passed down any number of > levels before being called, and any exceptions raised > in it will be propagated back up through all of them. > > > The other problem with thunks is that once we think of them as the > > anonymous functions they are, we're pretty much forced to say that a > > return statement in a thunk returns from the thunk rather than from > > the containing function. > > Urg, you're right. Unless return is turned into an > exception in that case. And then I suppose break and > return (and yield?) will have to follow suit. But wasn't that exactly what you were trying to avoid? :-) > I'm just trying to think how Smalltalk handles this, > since it must have a similar problem, but I can't > remember the details. > > > every > > local variable used or set in the thunk would have to become a 'cell' > > . Cells > > slow down access somewhat compared to regular local variables. > > True, but is the difference all that great? It's > just one more C-level indirection, isn't it? Alas not. It becomes a call to PyCell_Set() or PyCell_Get(). > > we'll want variables > > *assigned to* by the thunk also to be shared with the containing > > function, > > Agreed. We'd need to add a STORE_CELL bytecode or > something for this. This actually exists -- it is used for when an outer function stores into a local that it shares with an inner function. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
Shane Hathaway wrote: "Block statements provide a mechanism for encapsulating patterns of structure. Code inside the block statement runs under the control of an object called a block iterator. Simple block iterators execute code before and after the code inside the block statement. Block iterators also have the opportunity to execute the controlled code more than once (or not at all), catch exceptions, or receive data from the body of the block statement. That actually looks pretty reasonable. Hmmm. "Patterns of structure." Maybe we could call it a "struct" statement. struct opening(foo) as f: ... Then we could confuse both C *and* Ruby programmers at the same time! :-) [No, I don't really mean this. I actually prefer "block" to this.] -- Greg Ewing, Computer Science Dept, +--+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | [EMAIL PROTECTED] +--+ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: anonymous blocks
> If the use of block-statements becomes common for certain > tasks such as opening files, it seems to me that people are > going to encounter their use around about the same time > they encounter for-statements. We need *something* to > tell these people to enable them to understand the code > they're reading. > > Maybe it would be sufficient just to explain the meanings > of those particular uses, and leave the full general > explanation as an advanced topic. Right. The block statement is a bit like a chameleon: it adapts its meaning to the generator you supply. (Or maybe it's like a sewer: what you get out of it depends on what you put into it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 340 - possible new name for block-statement
> Far too close to the "for" loop, IMHO. I read that, I'd have to remind > myself every time, "now, which one is it that can receive values passed > back in: for ... in, or in ... for?" Whoa! Read the PEP closely. Passing a value back to the iterator (using "continue EXPR") is supported both in the for-loop and in the block-statement; it's new syntax so there's no backwards compatibility issue. The real difference is that when a for-loop is exited through a break, return or exception, the iterator is left untouched; but when the same happens in a block-statement, the iterator's __exit__ or __error__ method is called (I haven't decided what to call it). > Another possibility just occurred to me. How about "using"? Blah. I'm beginning to like block just fine. With using, the choice of word for the generator name becomes iffy IMO; and it almost sounds like it's a simple renaming: "using X as Y" could mean "Y = X". -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Re: PEP 340 - possible new name for block-statement
[Nicolas Fleury] > I would prefer something that would be > understandable for a newbie's eyes, even if it fits more with common > usage than with the real semantics behind it. For example a Boost-like > keyword like: > > scoped EXPR as VAR: > BLOCK Definitely not. In too many languages, a "scope" is a new namespace, and that's exactly what a block (by whichever name) is *not*. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com