Re: [Python-Dev] Keyword-only parameters
I personally find that keyword only arguments make for nicer APIS. It also makes subclassing easier because you are free to add new positional arguments later. Especially for boolean arguments, I think keyword only is a great API choice. On Tue, Apr 14, 2015 at 2:06 PM, Steven D'Aprano wrote: > On Tue, Apr 14, 2015 at 01:40:40PM -0400, Eric V. Smith wrote: > > > But, I don't see a lot of keyword-only parameters being added to stdlib > > code. Is there some position we've taken on this? Barring someone saying > > "stdlib APIs shouldn't contain keyword-only params", I'm inclined to > > make numeric_owner keyword-only. > > I expect that's because keyword-only parameters are quite recent (3.x > only) and most of the stdlib is quite old. > > Keyword-only feels right for this to me too. > > -- > Steve > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] who must makes FOR loop quicker
The iterator is not revaluated, instead, it is constructing a single
iterator, in this case a list_iterator. The list_iterator looks at the
underyling list to know how to iterate so when you mutate the underlying
list, the list_iterator sees that. This does not mee the expression used to
generate the iterator was re-evaluated.
On Wed, Aug 5, 2015 at 11:25 AM, John Doe wrote:
> To pass by reference or by copy of - that is the question from hamlet.
> ("hamlet" - a community of people smaller than a village python3.4-linux64)
>
> xlist = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
> i = 0
> for x in xlist:
> print(xlist)
> print("\txlist[%d] = %d" % (i, x))
> if x%2 == 0 :
> xlist.remove(x)
> print(xlist, "\n\n")
> i = i + 1
>
> So, catch the output and help me, PLEASE, improve the answer:
> Does it appropriate ALWAYS REevaluate the terms of the expression list in
> FOR-scope on each iteration?
> But if I want to pass ONCE a copy to FOR instead of a reference (as seen
> from an output) and reduce unreasonable reevaluation, what I must to do for
> that?
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] Daily reference leaks (d7e490db8d54): sum=61494
Sorry about introducing this. Where can I subscribe to these automated emails. Also, how do I go about running this locally? On default I tried running `./python -m test -l test_capi` did not print anything about leaks. I think that using `object.__new__` as a decorator here is the same as subclassing object, overriding __new__ and then making a call to `super().__new__` so I would imagine this bug could appear in less "clever" situations. I would love to help fix this issue; Benjamin, you mentioned that you think that maybe all heaptypes should have gc, do you have a suggestion on where I can look in the code to try to make this change? On Wed, Oct 21, 2015 at 11:53 AM, Random832 wrote: > Raymond Hettinger writes: > > Thanks for hunting this down. I had seen the automated reference leak > > posts but didn't suspect that a pure python class would have caused > > the leak. > > > > I'm re-opening > > https://mail.python.org/pipermail/python-dev/2015-October/141993.html > > and will take a look at it this weekend. If I don't see an obvious > > fix, I'll revert Joe's patch until a correct patch is supplied and > > reviewed. > > If a pure python class can cause a reference leak, doesn't that mean it > is only a symptom rather than the real cause? Or is it that the use of > @object.__new__ is considered "too clever" to be worth fixing? > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is there a reference manual for Python bytecode?
The number and meaning of the arguments are documented in the dis module: https://docs.python.org/3.6/library/dis.html On Sat, Dec 26, 2015 at 5:20 PM, Erik wrote: > Hi. > > Looking at ceval.c and peephole.c, there is - of course - lots of specific > hard-coded knowledge of the bytecode (e.g., number of operands and other > attributes). I'd like to experiment at this level, but I can't seem to find > a reference for the bytecode. > > Is there the equivalent of something like the ARM ARM(*) for Python > bytecode? I can read Python or C code if it's encoded that way, but I'm > looking for something that's a bit more immediate than deciphering what an > interpreter or optimizer is trying to do (i.e., some sort of table layout > or per-opcode set of attributes). > > BR, > E. > > (*) > http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0406c/index.html > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is there a reference manual for Python bytecode?
All arguments are 2 bytes, if there needs to be more, EXTENDED_ARG is used On Sat, Dec 26, 2015 at 5:51 PM, Erik wrote: > Hi Joe, > > On 26/12/15 22:36, Joe Jevnik wrote: > >> The number and meaning of the arguments are documented in the dis >> module: https://docs.python.org/3.6/library/dis.html >> > > OK - I *did* find that, but perhaps didn't immediately understand what it > was telling me. > > So, something documented as "OP_CODE" is a 1-byte op, something documented > as "OP_CODE(foo)" is a 2-byte op - and unless I missed one, there are no > 3-byte ops? > > Thanks, > E. > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bitwise operations for bytes and bytearray
You want to put the `xor` method in the `nb_xor` field of the `PyNumberMethods` structure that lives in the `tp_as_number` field of the bytes type object. Two things I noticed in a quick pass: you might want to add some type checking around the case where `a` or `b` is not a `PyByteArray` object. Also, variable length arrays are added in C99, so you will need to manually allocate the `raw_*` arrays in the heap. This seems like a cool feature! On Thu, Jan 7, 2016 at 5:26 PM, Blake Griffith wrote: > Hi! > > I'm interested in adding the functionality to do something like: > > >>> b'a' ^ b'b' > b'\x03' > > > Instead of the good ol' TypeError. > > I think both bytes and bytearray should support all the bitwise operations. > > I've never hacked on cpython before. I'm starting by just trying to add > xor to bytearray. I have a ByteArray_Xor function that I think should do > this here > https://github.com/cowlicks/cpython/commit/d6dddb11cdb33032b39dcb9dfdaa7b10d4377b5f > > But I'm not sure how to hook this in to the rest of cypython. I tried > adding it where bytearray_as_sequence is declared in this > bytearrayobject.c file. But that gave me compiler warnings and broke things. > > So now that I have this ByteArray_Xor function, how do I make it be > bytearray.__xor___? > > Thanks! > > Blake > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making sure dictionary adds/deletes during iteration always raise exception
> Is it possible to add a key, triggering a resize of the dict, then remove one, and continue iterating through the old (deallocated) memory? You can add and remove keys between calling next which would resize the dictionary; however, it will not iterate through uninitialized memory. The dictiter holds the current index and each time next is called it goes directly to ma_keys->dk_entries[saved_index] or ma_values[saved_index] On Tue, Dec 13, 2016 at 12:55 PM, Chris Angelico wrote: > On Wed, Dec 14, 2016 at 4:48 AM, Guido van Rossum > wrote: > > IIUC the private version gets updated every time the dict gets modified > -- > > but what we need here should only trigger when a key is added or removed, > > not when a value is updated. > > Is it possible to add a key, triggering a resize of the dict, then > remove one, and continue iterating through the old (deallocated) > memory? If so, that could potentially cause a crash. > > ChrisA > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > joe%40quantopian.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generator objects and list comprehensions?
List, set, and dict comprehensions compile like:
# input
result = [expression for lhs in iterator_expression]
# output
def comprehension(iterator):
out = []
for lhs in iterator:
out.append(expression)
return out
result = comprehension(iter(iterator_expression))
When you make `expression` a `yield` the compiler thinks that
`comprehension` is a generator function instead of a normal function.
We can manually translate the following comprehension:
result = [(yield n) for n in (0, 1)]
def comprehension(iterator):
out = []
for n in iterator:
# (yield n) as an expression is the value sent into this generator
out.append((yield n))
return out
result = comprehension(iter((0, 1)))
We can see this in the behavior of `send` on the resulting generator:
In [1]: g = [(yield n) for n in (0, 1)]
In [2]: next(g)
Out[2]: 0
In [3]: g.send('hello')
Out[3]: 1
In [4]: g.send('world')
---
StopIteration Traceback (most recent call last)
in ()
> 1 g.send('world')
StopIteration: ['hello', 'world']
The `return out` gets translated into `raise StopIteration(out)` because
the code is a generator.
The bytecode for this looks like:
In [5]: %%dis
...: [(yield n) for n in (0, 1)]
...:
1 0 LOAD_CONST 0 ( at
0x7f4bae68eed0, file "", line 1>)
3 LOAD_CONST 1 ('')
6 MAKE_FUNCTION0
9 LOAD_CONST 5 ((0, 1))
12 GET_ITER
13 CALL_FUNCTION1 (1 positional, 0 keyword pair)
16 POP_TOP
17 LOAD_CONST 4 (None)
20 RETURN_VALUE
.
---
1 0 BUILD_LIST 0
3 LOAD_FAST0 (.0)
>>6 FOR_ITER13 (to 22)
9 STORE_FAST 1 (n)
12 LOAD_FAST1 (n)
15 YIELD_VALUE
16 LIST_APPEND 2
19 JUMP_ABSOLUTE6
>> 22 RETURN_VALUE
In `` you can see us create the function, call `iter` on
`(0, 1)`, and then call `(iter(0, 1))`. In `` you can
see the loop like we had above, but unlike a normal comprehension we have a
`YIELD_VALUE` (from the `(yield n)`) in the comprehension.
The reason that this is different in Python 2 is that list comprehension,
and only list comprehensions, compile inline. Instead of creating a new
function for the loop, it is done in the scope of the comprehension. For
example:
# input
result = [expression for lhs in iterator_expression]
# output
result = []
for lhs in iterator_expression:
result.append(lhs)
This is why `lhs` bleeds out of the comprehension. In Python 2, adding
making `expression` a `yield` causes the _calling_ function to become a
generator:
def f():
[(yield n) for n in range(3)]
# no return
# py2
>>> type(f())
generator
# py3
>> type(f())
NoneType
In Python 2 the following will even raise a syntax error:
In [5]: def f():
...: return [(yield n) for n in range(3)]
...:
...:
File "", line 2
return [(yield n) for n in range(3)]
SyntaxError: 'return' with argument inside generator
Generator expressions are a little different because the compilation
already includes a `yield`.
# input
(expression for lhs in iterator_expression)
# output
def genexpr(iterator):
for lhs in iterator:
yield expression
You can actually abuse this to write a cute `flatten` function:
`flatten = lambda seq: (None for sub in seq if (yield from sub) and False)`
because it translates to:
def genexpr(seq):
for sub in seq:
# (yield from sub) as an expression returns the last sent in value
if (yield from sub) and False:
# we never enter this branch
yield None
That was a long way to explain what the problem was. I think that that
solution is to stop using `yield` in comprehensions because it is
confusing, or to make `yield` in a comprehension a syntax error.
On Wed, Jan 25, 2017 at 12:38 AM, Craig Rodrigues
wrote:
> Hi,
>
> Glyph pointed this out to me here: http://twistedmatrix.
> com/pipermail/twisted-python/2017-January/031106.html
>
> If I do this on Python 3.6:
>
> >> [(yield 1) for x in range(10)]
> at 0x10cd210f8>
>
> If I understand this: https://docs.python.org/
> 3/reference/expressions.html#list-displays
> then this is a list display and should give a list, not a generator object.
> Is there a bug in Python, or does the documentation need to be updated?
>
> --
> Craig
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> joe%40quantopian.com
>
>
___
Python-D
Re: [Python-Dev] Unexpected bytecode difference
As a general rule, you should not expect the bytecode to be the same
between different versions of CPython, including minor version changes. For
example, the instructions for dictionary literals are different in 3.4,
3.5, and 3.6.
On Fri, Jan 19, 2018 at 6:54 PM, Victor Stinner
wrote:
> Python bytecode format changed deeply in Python 3.6. It now uses
> regular units of 2 bytes, instead of 1 or 3 bytes depending if the
> instruction has an argument.
>
> See for example https://bugs.python.org/issue26647 "wordcode".
>
> But CALL_FUNCTION bytecode also evolved.
>
> Victor
>
> 2018-01-20 0:46 GMT+01:00 Alexander Belopolsky <
> alexander.belopol...@gmail.com>:
> > I have encountered the following difference between Python 3 and 2:
> >
> > (py3)
> compile('xxx', '<>', 'eval').co_code
> > b'e\x00S\x00'
> >
> > (py2)
> compile('xxx', '<>', 'eval').co_code
> > 'e\x00\x00S'
> >
> > Note that 'S' (the code for RETURN_VALUE) and a zero byte are swapped
> > in Python 2 compared to Python 3. Is this change documented
> > somewhere?
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> victor.stinner%40gmail.com
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> joe%40quantopian.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] inconsistency in annotated assigned targets
Currently there are many ways to introduce variables in Python; however, only a few allow annotations. I was working on a toy language and chose to base my syntax on Python's when I noticed that I could not annotate a loop iteration variable. For example: for x: int in range(5): ... This led me to search for other places where new variables are introduced and I noticed that the `as` target of a context manager cannot have an annotation. In the case of a context manager, it would probably need parenthesis to avoid ambiguity with a single-line with statement, for example: with ctx as (variable: annotation): body Finally, you cannot annotate individual members of a destructuring assignment like: a: int, b: int, c: int = 1, 2, 3 Looking at the grammar, these appear to be `expr` or `exprlist` targets. One change may be to allow arbitrary expressions to have an annotation . This would be a small change to the grammar but would potentially have a large effect on the language or static analysis tools. I am posting on the mailing list to see if this is a real problem, and if so, is it worth investing any time to address it. I would be happy to attempt to fix this, but I don't want to start if people don't want the change. Also, I apologize if this should have gone to python-idea; this feels somewhere between a bug report and implementation question more than a new feature so I wasn't sure which list would be more appropriate. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] inconsistency in annotated assigned targets
Thank you for the clarification! I should have looked through the PEPs first. On Thu, Jan 25, 2018 at 10:14 PM, Guido van Rossum wrote: > PEP 526 has this in the "Rejected/Postponed Proposals" section: > > - **Allow annotations in** ``with`` **and** ``for`` **statement:** > This was rejected because in ``for`` it would make it hard to spot the > actual > iterable, and in ``with`` it would confuse the CPython's LL(1) parser. > > > On Thu, Jan 25, 2018 at 3:17 PM, Jelle Zijlstra > wrote: > >> >> >> 2018-01-25 15:00 GMT-08:00 Joe Jevnik via Python-Dev < >> python-dev@python.org>: >> >>> Currently there are many ways to introduce variables in Python; however, >>> only a few allow annotations. I was working on a toy language and chose to >>> base my syntax on Python's when I noticed that I could not annotate a loop >>> iteration variable. For example: >>> >>> for x: int in range(5): >>> ... >>> >>> This led me to search for other places where new variables are >>> introduced and I noticed that the `as` target of a context manager cannot >>> have an annotation. In the case of a context manager, it would probably >>> need parenthesis to avoid ambiguity with a single-line with statement, for >>> example: >>> >>> with ctx as (variable: annotation): body >>> >>> Finally, you cannot annotate individual members of a destructuring >>> assignment like: >>> >>> a: int, b: int, c: int = 1, 2, 3 >>> >>> Looking at the grammar, these appear to be `expr` or `exprlist` targets. >>> One change may be to allow arbitrary expressions to have an annotation . >>> This would be a small change to the grammar but would potentially have a >>> large effect on the language or static analysis tools. >>> >>> I am posting on the mailing list to see if this is a real problem, and >>> if so, is it worth investing any time to address it. I would be happy to >>> attempt to fix this, but I don't want to start if people don't want the >>> change. Also, I apologize if this should have gone to python-idea; this >>> feels somewhere between a bug report and implementation question more than >>> a new feature so I wasn't sure which list would be more appropriate. >>> >> I have written a fair amount of code with variable annotations, and I >> don't remember ever wanting to add annotations in any of the three contexts >> you mention. In practice, variable annotations are usually needed for >> class/instance variables and for variables whose type the type checker >> can't infer. The types of loop iteration variables and context manager >> assignment targets can almost always be inferred trivially. >> >> >>> >>> ___ >>> Python-Dev mailing list >>> Python-Dev@python.org >>> https://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: https://mail.python.org/mailma >>> n/options/python-dev/jelle.zijlstra%40gmail.com >>> >>> >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% >> 40python.org >> >> > > > -- > --Guido van Rossum (python.org/~guido) > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Register-based VM [Was: Possible performance regression]
METH_FASTCALL passing arguments on the stack doesn't necessarily mean it will be slow. In x86 there are calling conventions that read all the arguments from the stack, but the rest of the machine is register based. Python could also look at ABI calling conventions for inspiration, like x86-64 where some arguments up to a fixed amount are passed on the stack and the rest are passed on the stack. One thing that I am wondering is would Python want to use a global set of registers and a global data stack, or continue to have a new data stack (and now registers) per call stack. If Python switched to a global stack and global registers we may be able to eliminate a lot of instructions that just shuffle data from the caller's stack to the callee's stack. On Tue, Feb 26, 2019 at 4:55 PM Victor Stinner wrote: > Hum, I read again my old REGISTERVM.txt that I wrote a few years ago. > > A little bit more context. In my "registervm" fork I also tried to > implement further optimizations like moving invariants out of the > loop. Some optimizations could change the Python semantics, like > remove "duplicated" LOAD_GLOBAL whereas the global might be modified > in the middle. I wanted to experiment such optimizations. Maybe it was > a bad idea to convert stack-based bytecode to register-based bytecode > and experiment these optimizations at the same time. > > Victor > > Le mar. 26 févr. 2019 à 22:42, Victor Stinner a > écrit : > > > > No, I wasn't aware of this project. My starting point was: > > > > http://static.usenix.org/events/vee05/full_papers/p153-yunhe.pdf > > Yunhe Shi, David Gregg, Andrew Beatty, M. Anton Ertl, 2005 > > > > See also my email to python-dev that I sent in 2012: > > https://mail.python.org/pipermail/python-dev/2012-November/122777.html > > > > Ah, my main issue was my implementation is that I started without > > taking care of clearing registers when the stack-based bytecode > > implicitly cleared a reference (decref), like "POP_TOP" operation. > > > > I added "CLEAR_REG" late in the development and it caused me troubles, > > and the "correct" register-based bytecode was less efficient than > > bytecode without CLEAR_REG. But my optimizer was very limited, too > > limited. > > > > Another implementation issue that I had was to understand some > > "implicit usage" of the stack like try/except which do black magic, > > whereas I wanted to make everything explicit for registers. I'm > > talking about things like "POP_BLOCK" and "SETUP_EXCEPT". In my > > implementation, I kept support for stack-based bytecode, and so I had > > some inefficient code and some corner cases. > > > > My approach was to convert stack-based bytecode to register-based > > bytecode on the fly. Having both in the same code allowed to me run > > some benchmarks. Maybe it wasn't the best approach, but I didn't feel > > able to write a real compiler (AST => bytecode). > > > > Victor > > > > Le mar. 26 févr. 2019 à 21:58, Neil Schemenauer > a écrit : > > > > > > On 2019-02-26, Victor Stinner wrote: > > > > I made an attempt once and it was faster: > > > > https://faster-cpython.readthedocs.io/registervm.html > > > > > > Interesting. I don't think I have seen that before. Were you aware > > > of "Rattlesnake" before you started on that? It seems your approach > > > is similar. Probably not because I don't think it is easy to find. > > > I uploaded a tarfile I had on my PC to my web site: > > > > > > http://python.ca/nas/python/rattlesnake20010813/ > > > > > > It seems his name doesn't appear in the readme or source but I think > > > Rattlesnake was Skip Montanaro's project. I suppose my idea of > > > unifying the local variables and the registers could have came from > > > Rattlesnake. Very little new in the world. ;-P > > > > > > Cheers, > > > > > > Neil > > > > > > > > -- > > Night gathers, and now my watch begins. It shall not end until my death. > > > > -- > Night gathers, and now my watch begins. It shall not end until my death. > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/joe%40quantopian.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inplace operations for PyLong objects
Is it true that checking for refcount == 1 is enough? What if a user wrote: args = (compute_integer(), 5) # give away args to someone int.__iadd__(*args) here `args[0]` still has refcount=1 because only `args` owns this integer. On Fri, Sep 1, 2017 at 4:19 PM, Jelle Zijlstra wrote: > > > 2017-09-01 13:05 GMT-07:00 Chris Barker : > >> On Thu, Aug 31, 2017 at 5:12 PM, Antoine Pitrou >> wrote: >> >>> I'm skeptical there are some programs out there that are limited by the >>> speed of PyLong inplace additions. >>> >> >> indeed, but that could be said about any number of operations. >> >> My question is -- how can the interpreter know if it can alter what is >> supposed to be an immutable in-place? If it's used only internally to a >> function, the it would be safe, but how to know that? >> > I believe Catalin's implementation checks if the object's refcount is 1. > If that is the case, it is safe to mutate it. > >> >> -CHB >> >> >> >> -- >> >> Christopher Barker, Ph.D. >> Oceanographer >> >> Emergency Response Division >> NOAA/NOS/OR&R(206) 526-6959 voice >> 7600 Sand Point Way NE (206) 526-6329 fax >> Seattle, WA 98115 (206) 526-6317 main reception >> >> chris.bar...@noaa.gov >> >> ___ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: https://mail.python.org/mailman/options/python-dev/jelle. >> zijlstra%40gmail.com >> >> > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > joe%40quantopian.com > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Inplace operations for PyLong objects
The string concat optimization happens in the interpreter dispatch for INPLACE_ADD On Fri, Sep 1, 2017 at 9:10 PM, Greg Ewing wrote: > Chris Angelico wrote: > >> This particular example is safe, because the arguments get passed >> individually - so 'args' has one reference, plus there's one more for >> the actual function call >> > > However, that's also true when you use the += operator, > so if the optimisation is to trigger at all in any useful > case, the refcount threshold needs to be set higher than > 1. > > Some experiments I did suggest that if you set it high > enough for x += y to trigger it, then it will also be > triggered in Joe's case. > > BTW, isn't there already a similar optimisation somewhere > for concatenating strings? Does it still exist? How does > it avoid this issue? > > -- > Greg > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/joe% > 40quantopian.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Helpers for dynamic bytecode generation
I think this probably belongs on python-list instead of python-dev because python-dev is for development _of_ python, not _with_ python. To answer your question though, there are a few tools that do this: - https://github.com/vstinner/bytecode - https://github.com/ll/codetransformer I am one of the authors of codetransformer, so I am more familiar with how to use that to generate bytecode, but the bytecode project may be easier for your use case. On Thu, Oct 24, 2019 at 8:59 PM Yonatan Zunger wrote: > Hi everyone, > > I've found myself recently writing Python code that dynamically generates > bytecode.¹ I now have yet another case where I'm having to do this, in > which my nice situation of being able to easily precompute all the jump > addresses no longer holds. So I'm starting to write a helper to make it > easy to write bytecode from Python, with its basic API calls being > write(opcode, arg) and nextLine(optional label). The argument can be an > int, name, local name, constant, label, etc., depending on the opcode, and > it maintains all the appropriate tables and finally dumps a code object at > the end. > > All of which is well and good and makes life much easier, but... I am > *not* looking forward to writing the logic that basically duplicates that > of assemble() in compile.c, of splitting all of this into basic blocks and > computing the correct jump positions and so on before finally dumping out > the bytecode. > > Has anyone already done this that people know of? (Searching the > Internetz didn't turn anything up) Failing that, to what extent is it > reasonable to either consider assemble() as some kind of sane API point > into compile.c, and/or add some new API in compile.h which implements all > of the stuff described above in C? > > (I'm fully expecting the answer to these latter questions to be "you have > got to be kidding me," but figured it was wiser to check than to reinvent > this particular wheel if it isn't necessary) > > Yonatan > > > > ¹ Not out of masochism, in case you're wondering; there was a real use > case. A storage system would receive a read request that specified a bunch > of (key, condition) pairs, where conditions where either return any value, > return an exact value, or return values in a range. It would then receive > between 1 and 1M (depending on the request parameters) candidate cells from > the underlying storage layers, each of which had a tuple of bytes as its > actual key values; it had to compare each of those tuples against the > request parameters, and yield the values which matched. Because it's an > inner loop and can easily be called 1M times, doing this in pure Python > slows things down by a lot. Because it's also only called once, doing some > really expensive overhead like synthesizing Python code and calling > compile() on it would also slow things down a lot. But converting a bunch > of (key, condition) pairs to a really efficient function from tuples of > bytes to bools was pretty easy. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/E57DTM65LFEROFZLHKRV442JPPFAWNJU/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TF5NLPS5DQVR3N75VHIXOGSK5APZMLGK/ Code of Conduct: http://python.org/psf/codeofconduct/
