[issue4134] Peephole optimization: returning a temp
New submission from David Turner <[EMAIL PROTECTED]>: This patch adds an optimization to the peephole optimizer for the common case of: def func(...): ... retval = ... return retval Before the patch, the compiler would generate STORE_FAST 3 LOAD_FAST 3 RETURN_VALUE The store and load are pointless, as the local 3 (or whatever) is never used again. This patch makes the peephole optimizer optimize them out. -- components: None files: peephole-return-temp.diff keywords: patch messages: 74853 nosy: novalis_dt severity: normal status: open title: Peephole optimization: returning a temp versions: Python 2.7 Added file: http://bugs.python.org/file11811/peephole-return-temp.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4134> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4134] Peephole optimization: returning a temp
David Turner <[EMAIL PROTECTED]> added the comment: The idiom appears at least 370 times in the standard library. And, while on its own it can't appear in a loop, a function that uses it might get called in a loop. You mention some sort of AST optimizer. I haven't really followed Python development enough to know anything about this. Is it worth bothering to do other work on the peepholer which might handle some more common cases? If not, is there a branch or something where I can look at the AST optimizer? ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4134> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4141] Dis docs on CALL_FUNCTION a bit unclear
New submission from David Turner <[EMAIL PROTECTED]>: This patch improves them. -- files: dis-doc-patch.diff keywords: patch messages: 74928 nosy: novalis_dt severity: normal status: open title: Dis docs on CALL_FUNCTION a bit unclear Added file: http://bugs.python.org/file11821/dis-doc-patch.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4141> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4141] Dis docs on CALL_FUNCTION a bit unclear
Changes by David Turner <[EMAIL PROTECTED]>: -- assignee: -> georg.brandl components: +Documentation nosy: +georg.brandl versions: +Python 2.7 ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4141> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4264] Patch: optimize code to use LIST_APPEND instead of calling list.append
New submission from David Turner <[EMAIL PROTECTED]>: This is a patch to Tom Lee's AST optimization branch. Python bytecode includes at least one operation which is not directly accessible from Python code: LIST_APPEND, which pops a value and a list off the stack and appends the value to the list. This is used in generating code for list comprehensions: [x for x in somelist] generates the following code for the body of the loop: ... LOAD_FAST 1#a local is generated to hold the new list LOAD_FAST 2 #the iteration variable, x LIST_APPEND ... Whereas if you were to try to write the comprehension directly, you would get: LOAD_FAST 1 LOAD_ATTR 0 #an index into the constant table: “append” LOAD_FAST 2 CALL_FUNCTION 1 #remove x and the append function from the top of the stack and push the result of the call POP_TOP This is much slower. In part, it’s the cost of doing the attribute lookup each time, which is why it’s common to see code like a = [] ap = a.append for x in …..: ap(x + …) return a But the function call is naturally slower than the simpler LIST_APPEND operation. The attached patch tries to determine statically if a local is all circumstances holds a list, and if so, generates LIST_APPEND whenever user code would call local.append. It’s not perfect — in particular, I could track local types on a more fine-grained level than per-function. But it’s a start. -- files: tlee-ast-optimize-appends.diff keywords: patch messages: 75525 nosy: novalis_dt severity: normal status: open title: Patch: optimize code to use LIST_APPEND instead of calling list.append Added file: http://bugs.python.org/file11946/tlee-ast-optimize-appends.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4264> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4264] Patch: optimize code to use LIST_APPEND instead of calling list.append
David Turner <[EMAIL PROTECTED]> added the comment: Neal Norwitz <[EMAIL PROTECTED]> added the comment: > > Interesting approach. I was surprised to see the change to the AST, > but I understand why you did it. I think if the AST optimization > approach works out well, we will want to have some more general > mechanism to communicate these optimization (or other!) hints to the > compiler. It would suck to have to modify the AST each time we > wanted to add a new optimization. You and Tom might have better > ideas for how best to achieve that. I really didn't want to have to change the AST. The problem was that there was a feature of the Python bytecode which was not representable in Python source code. I don't anticipate future optimizations requiring changes to the AST, because I now believe every important feature of the bytecode is representable in the AST. The one exception might be if we have a need for arbitrary jumps. I don't think that would be useful, but it might conceivably. In that case, we would need Goto and Label AST nodes. The thing that struck me as ugly was the idea that there's one object (the optimizer) that knows everything about all optimization operations. This seems like a great case for the visitor pattern. The optimizer ought to have a list of functions to call for each type of AST node, each with some private storage space. But I didn't want to make that kind of dramatic change without consulting with Tom. > I'll make some comments that would need to be addressed, but you might > want to wait making any changes depending on what approach you decide > to take. > > The most important missing piece is tests to show that the new code > works. > There are various whitespace issues. Both whitespace only changes > that should be avoided and indentation inconsistencies with the > existing code (or so it appears). The latter could be the way I'm > viewing the file or existing problems with tabs. Fixed, I think. The original code appears to be somewhat inconsistent between files in its uses of spaces/tabs, but I think I have now matched the style of each file. > In Python/optimize.c, you need to handle the case where the > PyDict_New() calls fail. It looks like currently an undetected error > can happen during construction. And on destruction it will crash > because the objects will be NULL when calling Py_DECREF. Fixed. > All calls like PyDict_SetItem(), PyInt_FromLong(), etc need to handle > errors. I'm not exactly sure which "etc" need to handle errors. Py*_As*? Anyway, I changed the ones you mentioned. > I'll need to study the code a lot more to see how well it behaves. > Tests would help a lot with that. I've added a few. Added file: http://bugs.python.org/file11957/tlee-ast-optimize-appends-2.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4264> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4264] Patch: optimize code to use LIST_APPEND instead of calling list.append
David Turner <[EMAIL PROTECTED]> added the comment: Actually, I just noticed a case where the code would do the wrong thing. I fixed it and added a test for it. Added file: http://bugs.python.org/file11958/tlee-ast-optimize-appends-3.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4264> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4264] Patch: optimize code to use LIST_APPEND instead of calling list.append
David Turner <[EMAIL PROTECTED]> added the comment: > FWIW, I see exposing bytecodes as an anti-pattern that locks in a > particular implementation in a way that makes it hard to change and > hard to port to other Python implementations. The current bound > method approach works fine. Don't really see enough payoff to > justify the extra code and maintenance. Bytecodes aren't exposed at the language level -- just during compilation. As far as I know, other implementations don't use any of the Python compiler mechanics, so there's no portability problem. If another compiler did use the Python AST, they could just compile the Append node kind to a method call with no loss of speed or clarity. Or they could just not use this optimization, if we refactor the ast optimizer as I propose to use the strategy pattern. As for the payoff, microbenchmarks show a big win. Also, appending to a list is a fairly common pattern. Our Plone-based codebase shows tens of thousands of instances of append. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4264> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4264] Patch: optimize code to use LIST_APPEND instead of calling list.append
David Turner <[EMAIL PROTECTED]> added the comment: > Obviously if there's another sufficiently good argument for the visitor > approach, I'm all ears. But again, if we do that I think we should do > it for the compiler as well. I'm not sure how much support such a > change would have. The argument is that it would be possible to enable or disable individual optimizations this way. For the compiler, there's no need for this, because there's only one thing to do per node type (although I suppose we could just pass that set of things into the node walker). Another argument against is that it would be harder to combine optimizations when that's relevant. I don't think it's worth worrying about until there are a dozen or so AST-level optimizations. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4264> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4327] Patch: simplify complex constant assignment statements
New submission from David Turner <[EMAIL PROTECTED]>: This patch adds functionality to the optimizer to simplify complex constant assignments like: a, (b, c) = d, e = 1, (2, 3) The simplification is: a = 1 d = 1 b, c = e = 2, 3 Of course, the simplified version is semantically identical. But the bytecode generated is faster, because there is less need to unpack tuples. Naturally, this only works on constants: a, b = 1, a is not the same as a = 1 b = a -- files: tlee-ast-optimize-multiassign.diff keywords: patch messages: 75889 nosy: novalis_dt severity: normal status: open title: Patch: simplify complex constant assignment statements Added file: http://bugs.python.org/file12012/tlee-ast-optimize-multiassign.diff ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4327] Patch: simplify complex constant assignment statements
David Turner <[EMAIL PROTECTED]> added the comment: Oh, and this also involved the creation of an additional statement type, unfortunately. The statement type, Seq, represents a sequence of statements. This is so that we can replace a single assign with multiple assigns. If that's no good, then we could do an If with test=1 instead, since the compiler knows to optimize that out. We could also go into the parent statement and replace the body, but that would require significant rejiggering of the optimizer, and significant additional complexity. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4327] Patch: simplify complex constant assignment statements
David Turner <[EMAIL PROTECTED]> added the comment: jhylton, keep in mind that this would require an additional "parent" argument to each function which takes a stmt. Do you think this added complexity is worth it? ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4327] Patch: simplify complex constant assignment statements
David Turner <[EMAIL PROTECTED]> added the comment: Sure, but that's an even bigger change. Every piece of code which modifies the AST would now also have to modify parent pointers. Having the pointers would make many things easier, but I didn't want to make a very invasive change like that without thinking it through thoroughly. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4327] Patch: simplify complex constant assignment statements
David Turner <[EMAIL PROTECTED]> added the comment: Everything in optimize.c is about modifying the AST (this is on tlee's ast optimization branch, in case I didn't mention earlier). Also, changing asdl_seq_SET would be a bad idea, since it is used for sequences of things other than stmt_tys. What if asdl_seq were instead a doubly-linked list? Then each node contains enough information to add and remove elements. asdl_seq_LEN would be slower, but it is rarely used. To be really slick, we could store the len of the seq in the first element's prev pointer as (len << 1) | 1. ___ Python tracker <[EMAIL PROTECTED]> <http://bugs.python.org/issue4327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39141] IDLE Clear function returns 256 on Mac OS Catalina
New submission from David Turner :
Trying to set up shortcut function to clear screen but its not working as
expected on my Mac OS Catalina -- below is txt from idle
import os
>>> cls= lambda: os.system('clear')
>>> cls()
256
--
messages: 358908
nosy: twiste...@gmail.com
priority: normal
severity: normal
status: open
title: IDLE Clear function returns 256 on Mac OS Catalina
type: behavior
versions: Python 3.8
___
Python tracker
<https://bugs.python.org/issue39141>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21248] BROWSER env var described inaccurately in webbrowser docs
New submission from David Turner: https://docs.python.org/2/library/webbrowser.html says "If the environment variable BROWSER exists, it is interpreted to override the platform default list of browsers, as a os.pathsep-separated list of browsers to try in order." This is not actually what happens; instead, it is prepended to the platform-default list of browsers. -- assignee: docs@python components: Documentation messages: 216393 nosy: docs@python, dturner-tw priority: normal severity: normal status: open title: BROWSER env var described inaccurately in webbrowser docs versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue21248> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
