[issue37019] using pathlib objects to create symlinks produces broken links

2019-05-23 Thread Shannon


New submission from Shannon :

when using pathlib objects to define src and dst for os.symlink (also relevant 
for Path(dst).symlink_to(src)), if the src path is not absolute, or relative to 
the directory the link is being created in, a broken link will be created.

example/

src = pathlib.Path('dir1/file')
dst = pathlib.Path('dir2/file')
os.symlink(src, dst) # or dst.symlink_to(src)

this will create a broken link in dir2, attempting to link to dir1/file, 
relative to dir2.

It seems to me, if src given is a pathlib object (relative to cwd), the linking 
process should be smart enough to point the link the created symlink to the 
right place.

os.symlink(src.absolute(), dst) works, but creates an absolute symlink which 
may not be desired behaviour.

My current workaround is:
os.symlink(os.path.relpath(src, dst.parent), dst)
which creates a working relative symlink as desired. I would suggest this 
should be the default behaviour of both os.symlink and 
pathlib.Path().symlink_to when a non-absolute path object is given as src.

Interestingly, src.relative_to(dst.parent) will raise a ValueError while 
os.path.relpath(src, dst.parent) correctly returns '../dir1/file'.
I also think Path().relative_to should be changed to match the behaviour of 
os.path.relpath here, but perhaps that is a separate issue.

--
components: Library (Lib)
messages: 343277
nosy: skeo
priority: normal
severity: normal
status: open
title: using pathlib objects to create symlinks produces broken links
type: behavior
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue37019>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37019] Create symlinks relative to cwd

2019-05-23 Thread Shannon


Shannon  added the comment:

I can see how this change would be backward incompatible, however the current 
behaviour seems inconsistent with the way pathlib functions otherwise. Within 
two lines of code, the same path object can be pointing to two completely 
different locations simply because it's being used as the src argument to 
create a symlimk. That's counter to my understanding of the whole point of 
pathlib.

With regard to relative_to, I've found this issue, which doesn't seem to have 
been touched since 2016. https://bugs.python.org/issue20012
I'm new here, what's the best way to bump an issue?

--

___
Python tracker 
<https://bugs.python.org/issue37019>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue29988] with statements are not ensuring that __exit__ is called if __enter__ succeeds

2021-12-10 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue29988>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46039] Break up the YIELD_FROM instruction.

2021-12-10 Thread Mark Shannon


New submission from Mark Shannon :

The YIELD_FROM instruction does three things:

* It sends a value to the sub-iterator
* It yields the value from the sub-iterator back up to its caller
* Loops back on itself

So we should implement this as:

SEND<--+
YIELD_VALUE|
JUMP_ABSOLUTE -+

Doing so would allow us to simplify gen.send and gen.throw as they wouldn't 
need all the special cases for 'yield from'.

Zero cost exception handling allows us to handle throw in the bytecode with no 
runtime overhead:

while True:
SEND -> exit
try:
YIELD_VALUE
except BaseException as ex:
sub_iterator.throw(ex)
exit:

--
assignee: Mark.Shannon
components: Interpreter Core
messages: 408232
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Break up the YIELD_FROM instruction.
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46039] Break up the YIELD_FROM instruction.

2021-12-10 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28260
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30035

___
Python tracker 
<https://bugs.python.org/issue46039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-14 Thread Mark Shannon


New submission from Mark Shannon :

By "stats" I mean the internal numbers gathered by the VM for performance and 
debugging. This has nothing to do with any statistics module.

Currently various parts of the VM gather stats: the GC, dicts, the bytecode 
interpreter, type lookup cache, etc.

These stats have various compile time flags to turn them on and off. They have 
differing ways to display the stats, and no unified way to gather stats across 
different runs.

For the specialization stats we dump stats, which we can parse to collate stats 
across runs.

We should:
1. Add a --with-pystats config flag (like with-pydebug) to turn on stat 
gathering at build time.
2. Convert the other stats to the format used by the specialization stats, so 
all stats can be parsed and collated.

--
messages: 408543
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Unify handling of stats in the CPython VM

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44525] Implement CALL_FUNCTION adaptive interpreter optimizations

2021-12-14 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 9f8f45144b6f0ad481e80570538cce89b414f7f9 by Mark Shannon in 
branch 'main':
bpo-44525: Split calls into PRECALL and CALL (GH-30011)
https://github.com/python/cpython/commit/9f8f45144b6f0ad481e80570538cce89b414f7f9


--

___
Python tracker 
<https://bugs.python.org/issue44525>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44525] Implement CALL_FUNCTION adaptive interpreter optimizations

2021-12-14 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +28329
pull_request: https://github.com/python/cpython/pull/30107

___
Python tracker 
<https://bugs.python.org/issue44525>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45292] Implement PEP 654: Exception Groups

2021-12-14 Thread Mark Shannon


Mark Shannon  added the comment:

PR 29581 resulted in a 1% slowdown, which is not terrible, but code not using 
except* should not be slowed down at all.

IMO, the way to avoid the slowdown is to implement except* using the existing 
instruction set (perhaps with a few minor additions)

We already implement try-finally, named except blocks and with statements 
without any complex bytecodes (except perhaps WITH_EXCEPT_START).

These used to involve a lot of state and more complex bytecodes. So it is 
possible to make these simplifications,
but it does take work.


There are a number of techniques we can use:

If any state is needed, push it to the stack as we do with `ctx.__exit__` in 
the with statement, and when pushing f_lasti in exception handlers.
Duplicate code paths when the semantics differ in different cases, as we do for 
finally blocks.
If anything is too complex to handle on the stack, put it in a temporary 
variable.
Be liberal in your use of virtual try-excepts (SETUP_FINALLY, POP_FINALLY 
pairs), as zero-cost exception handling should keep the cost down.


It may be too late for this advice, but if I were writing the `except*` 
implementation from scratch, I would:

1. Sketch out the pseudo Python that a try-except* would map to. This is a good 
opportunity to discover any design bugs that might result in undesirable 
behavior for corner cases.

2. Implement the translation in the compiler, not worrying about any redundancy 
or inefficiency, just correctness.

3. Look to improve the above, either in the compiler front-end, or by replacing 
inefficient code patterns in the back-end. Handling the optimization in the 
backend has the advantage that other code might benefit as well.

--

___
Python tracker 
<https://bugs.python.org/issue45292>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46039] Break up the YIELD_FROM instruction.

2021-12-15 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 0b50a4f0cdee41a18fb4ba6e75569f9cfaceb39e by Mark Shannon in 
branch 'main':
bpo-46039: Split yield from in two (GH-30035)
https://github.com/python/cpython/commit/0b50a4f0cdee41a18fb4ba6e75569f9cfaceb39e


--

___
Python tracker 
<https://bugs.python.org/issue46039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-15 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28338
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30116

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44525] Implement CALL_FUNCTION adaptive interpreter optimizations

2021-12-15 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 3a60bfef49b3324660a615a8e6d10710e5f669d9 by Mark Shannon in 
branch 'main':
bpo-44525: Specialize for calls to type and other builtin classes with 1 
argument. (GH-29942)
https://github.com/python/cpython/commit/3a60bfef49b3324660a615a8e6d10710e5f669d9


--

___
Python tracker 
<https://bugs.python.org/issue44525>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-15 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 342b93f9f28746abb7b221a61d5a9b26ccbb395a by Mark Shannon in 
branch 'main':
bpo-46072: Add --with-pystats configure option to simplify gathering of VM 
stats (GH-30116)
https://github.com/python/cpython/commit/342b93f9f28746abb7b221a61d5a9b26ccbb395a


--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46006] [subinterpreter] _PyUnicode_EqualToASCIIId() issue with subinterpreters

2021-12-15 Thread Mark Shannon


Mark Shannon  added the comment:

The problem here is that different sub-interpreters have different strings for 
the same Python string.

Unless sub-interpreters are fully independent, and they cannot be due to 
limitations imposed by the stable API, then all sub-interpreters must share the 
same poll of strings.

Since the only object reachable from a string is the `str` object (which is a 
static global object `PyUnicode_Type`), then the invariant that no object that 
is unique to one sub-interpreter can be reached from another sub-interpreter 
remains valid if strings are shared. I.e. there is no reason not to share 
strings.

As Victor points out, there is no bug in 3.9 because interned strings are 
common across all interpreter. We should revert that behavior.

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46006>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46097] Split function versions into 1-0xffff and 0x1000+ regions

2021-12-16 Thread Mark Shannon


New submission from Mark Shannon :

Because functions are mutable, specifically because the __code__ attribute is 
mutable, we need to version functions when specializing.

However, some specializations (for special methods mainly) only have space for 
16 bit versions.

It is likely that programs will have more than 2**16 functions versions, but it 
is much less likely that they will have more than 2**16 versions of special 
methods.

We should partition the version space into 1-0x for use by special methods 
and 0x1000+ for use by other methods.

See https://github.com/python/cpython/pull/30129 for an example of why this is 
needed.

--
components: Interpreter Core
messages: 408686
nosy: Mark.Shannon, brandtbucher
priority: normal
severity: normal
status: open
title: Split function versions into 1-0x and 0x1000+ regions
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46097>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45829] Remove C stack use by specializing BINARY_SUBSCR, STORE_SUBSCR, LOAD_ATTR, and STORE_ATTR

2021-12-16 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 62a8a0c5223f750e22ee381d3cfbdb718cf1cc93 by Brandt Bucher in 
branch 'main':
bpo-45829: Check `__getitem__`'s version for overflow before specializing 
(GH-30129)
https://github.com/python/cpython/commit/62a8a0c5223f750e22ee381d3cfbdb718cf1cc93


--

___
Python tracker 
<https://bugs.python.org/issue45829>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-16 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +28357
pull_request: https://github.com/python/cpython/pull/30139

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-16 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 4506bbede1644e985991884964b43afa7ee6f609 by Mark Shannon in 
branch 'main':
bpo-46072: Document --enable-stats option. (GH-30139)
https://github.com/python/cpython/commit/4506bbede1644e985991884964b43afa7ee6f609


--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-16 Thread Mark Shannon


Mark Shannon  added the comment:

The --enable-stats option is for CPython development and shouldn't be turned on 
for a release version, so I'm not really concerned about people attacking their 
own machines.

--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-16 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +28363
pull_request: https://github.com/python/cpython/pull/30145

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-17 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +28386
pull_request: https://github.com/python/cpython/pull/30169

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45711] Simplify the interpreter's (type, val, tb) exception representation

2021-12-17 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 396b58345f81d4c8c5a52546d2288e666a1b9b8b by Irit Katriel in 
branch 'main':
bpo-45711: Remove type and traceback from exc_info (GH-30122)
https://github.com/python/cpython/commit/396b58345f81d4c8c5a52546d2288e666a1b9b8b


--

___
Python tracker 
<https://bugs.python.org/issue45711>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2021-12-17 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset efd6236d36b292c2c43540132c87cf8425e8d627 by Mark Shannon in 
branch 'main':
bpo-46072:  Add top level stats struct (GH-30169)
https://github.com/python/cpython/commit/efd6236d36b292c2c43540132c87cf8425e8d627


--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45923] Improve performance of sys.settracing based tools.

2022-01-03 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28578
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30364

___
Python tracker 
<https://bugs.python.org/issue45923>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46202] remove opcode POP_EXCEPT_AND_RERAISE

2022-01-04 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset a94461d7189d7f1147ab304a332c8684263dc17e by Irit Katriel in 
branch 'main':
bpo-46202: Remove opcode POP_EXCEPT_AND_RERAISE (GH-30302)
https://github.com/python/cpython/commit/a94461d7189d7f1147ab304a332c8684263dc17e


--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46202>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45609] Specialize STORE_SUBSCR

2022-01-04 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 7537f6008704b20e2d04a7ef1c0cfa34121cc5eb by Dennis Sweeney in 
branch 'main':
bpo-45609: More specialization stats for STORE_SUBSCR (GH-30193)
https://github.com/python/cpython/commit/7537f6008704b20e2d04a7ef1c0cfa34121cc5eb


--

___
Python tracker 
<https://bugs.python.org/issue45609>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44525] Implement CALL_FUNCTION adaptive interpreter optimizations

2022-01-05 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +28621
pull_request: https://github.com/python/cpython/pull/30415

___
Python tracker 
<https://bugs.python.org/issue44525>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43683] Handle generator (and coroutine) state in the bytecode.

2022-01-05 Thread Mark Shannon


Mark Shannon  added the comment:

Yes, most of it :)

We haven't implemented points 2 and 3, yet.

I'm in no hurry to implement 3. It would clean up `gen.throw` a lot, and break 
the dependency between that code and the interpreter, but it isn't urgent.

2 is more urgent. I think we need it to allow specialization of `FOR_ITER` for 
generators, so it should happen in the next few weeks.

--

___
Python tracker 
<https://bugs.python.org/issue43683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45256] Remove the usage of the C stack in Python to Python calls

2022-01-05 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 332e6b972567debfa9d8f3f9a4a966c7ad15eec9 by Brandt Bucher in 
branch 'main':
bpo-45256: Don't track the exact depth of each `InterpreterFrame` (GH-30372)
https://github.com/python/cpython/commit/332e6b972567debfa9d8f3f9a4a966c7ad15eec9


--

___
Python tracker 
<https://bugs.python.org/issue45256>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue44525] Implement CALL_FUNCTION adaptive interpreter optimizations

2022-01-05 Thread Mark Shannon


Mark Shannon  added the comment:

See https://github.com/faster-cpython/ideas/discussions/210

--

___
Python tracker 
<https://bugs.python.org/issue44525>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46031] add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE

2022-01-06 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 3db762db72cc0da938614b1e414abb1e12ca4094 by penguin_wwy in branch 
'main':
bpo-46031: add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE (GH-30019)
https://github.com/python/cpython/commit/3db762db72cc0da938614b1e414abb1e12ca4094


--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46031>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40222] "Zero cost" exception handling

2022-01-06 Thread Mark Shannon


Mark Shannon  added the comment:

Yes.

--
priority: release blocker -> 
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40222>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45256] Remove the usage of the C stack in Python to Python calls

2022-01-06 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue45256>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46039] Break up the YIELD_FROM instruction.

2022-01-06 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46166] Get "self" args or non-null co_varnames from frame object with C-API

2022-01-06 Thread Mark Shannon


Change by Mark Shannon :


--
nosy: +eric.snow

___
Python tracker 
<https://bugs.python.org/issue46166>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45923] Improve performance of sys.settracing based tools.

2022-01-06 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset e028ae99ecee671c0e8a3eabb829b5b2acfc4441 by Mark Shannon in 
branch 'main':
bpo-45923: Handle call events in bytecode (GH-30364)
https://github.com/python/cpython/commit/e028ae99ecee671c0e8a3eabb829b5b2acfc4441


--

___
Python tracker 
<https://bugs.python.org/issue45923>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46225] f_lasti behaves differently for lambdas returned from loops

2022-01-06 Thread Mark Shannon


Mark Shannon  added the comment:

Are you using `f_lasti == -1` as a proxy for "is this an actual call or a 
resumption after a yield"?

Ultimately PEP 669 will provide all the information you could need to 
differentiate different events.

In the meantime...

We now make all "call" events explicit in the bytecode using the new RESUME 
instruction.

That means the following is true for *all* call events:
  f_lasti >= 0
  code.co_code[f_lasti*2] == opcode.opmap["RESUME"]

The oparg of the RESUME instruction describes the kind of "call" event.
Given `oparg = code.co_code[f_lasti*2+1]`, then
`oparg == 0` implies that the event is an actual call.

See https://docs.python.org/3.11/library/dis.html#opcode-RESUME


Does that help?

--

___
Python tracker 
<https://bugs.python.org/issue46225>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46031] add POP_JUMP_IF_NOT_NONE and POP_JUMP_IF_NONE

2022-01-06 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46031>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-10 Thread Mark Shannon


Mark Shannon  added the comment:

Thanks, Ned.

--
assignee:  -> Mark.Shannon
priority: normal -> release blocker
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-10 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28716
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30513

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-10 Thread Mark Shannon


Mark Shannon  added the comment:

Pablo, I've marked this as a release blocker so this bug doesn't get into 
3.11a4.

--
keywords:  -patch
nosy: +pablogsal
stage: patch review -> 

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-10 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset ec0c392f34ee2474ceacf66881f05546b540e2d1 by Mark Shannon in 
branch 'main':
bpo-46314: Remove extra RESUME when compiling a lamdba. (GH-30513)
https://github.com/python/cpython/commit/ec0c392f34ee2474ceacf66881f05546b540e2d1


--

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-10 Thread Mark Shannon


Mark Shannon  added the comment:

Ok, I'll add a news item in that case.

--

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-10 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28718
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30515

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46329] Split up the CALL_NO_KW and CALL_KW instructions.

2022-01-10 Thread Mark Shannon


New submission from Mark Shannon :

Most calls (not those with *args and/or **kwargs) are compiled to the 
`CALL_NO_KW` and `CALL_KW` instructions, possibly with a `PRECALL_METHOD` 
prefix.

We want to unify the `CALL_NO_KW` and `CALL_KW` instructions into a single 
`CALL` instruction and add two new instructions: `PRECALL_FUNCTION` and 
`KW_NAMES`.
All call sequences would start with `PRECALL_FUNCTION` or `PRECALL_METHOD`. 
`PRECALL_METHOD` would continue to pair with `LOAD_METHOD` as it does now. 
`PRECALL_FUNCTION` would effectively be a no-op, but would be needed to set up 
internal interpreter variables.

`CALL_NO_KW` would become `CALL` and `CALL_KW` would become `KW_NAMES; CALL`.

Why?

Specializing calls is an important optimization, but calls are complicated and 
we want to specialized for both the type of the callable and the shape of the 
call.

By breaking up calls in this way, we can specialize for the type and for the 
shape mostly independently.
We can specialize for classes, bound-methods and other objects that ultimately 
call a Python function in the `PRECALL` instruction and specialize for the 
shape of the arguments in the `CALL` instruction.


See https://github.com/faster-cpython/ideas/discussions/210 for more rationale.

--
assignee: Mark.Shannon
messages: 410209
nosy: Mark.Shannon, kj
priority: normal
severity: normal
status: open
title: Split up the CALL_NO_KW and CALL_KW instructions.
type: performance
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46329>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-10 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28720
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30518

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-10 Thread Mark Shannon


Change by Mark Shannon :


--
assignee:  -> Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-11 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset bd04fac7eb929cd11ab6985deb61d9780447fbff by Mark Shannon in 
branch 'main':
bpo-46331: Do not set line number of instruction storing doc-string. (GH-30518)
https://github.com/python/cpython/commit/bd04fac7eb929cd11ab6985deb61d9780447fbff


--

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-11 Thread Mark Shannon


Mark Shannon  added the comment:

Ned, does this fix work for you?

--
status: pending -> open

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46344] 3.11: except/else/if/raise traced incorrectly

2022-01-11 Thread Mark Shannon


Change by Mark Shannon :


--
nosy: +iritkatriel

___
Python tracker 
<https://bugs.python.org/issue46344>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46356] [C API] Enforce usage of PyFrame_GetBack()

2022-01-12 Thread Mark Shannon


Mark Shannon  added the comment:

f_lineno is not part of the API. No field in PyFrameObject is.

I don't know how we can stop people from using them though.

If they don't know better than pulling data out of undocumented internal 
struct, then I'm not sure an underscore prefix is going to help.
It won't do any harm, though.

--

___
Python tracker 
<https://bugs.python.org/issue46356>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46355] [C API] Document PyFrameObject and PyThreadState changes and explain how to port code to Python 3.11

2022-01-12 Thread Mark Shannon


Mark Shannon  added the comment:

Are there any C programming guides we can link to, that explain API use?
I would hope that competent C programmers would know not to read or write to 
undocumented fields. But if they come from a Python background, that might not 
be obvious.

If the advice for something is "use Python", we should probably add an API 
function.

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46355>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-12 Thread Mark Shannon


Change by Mark Shannon :


--
assignee: Mark.Shannon -> 
keywords:  -patch
stage: resolved -> 

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46314] Stray RESUME opcode for unused lambda

2022-01-12 Thread Mark Shannon


Mark Shannon  added the comment:

The news item was added in PR 30515

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46314>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45786] Avoid allocating when exiting frame; it may be unsafe.

2022-01-13 Thread Mark Shannon


Mark Shannon  added the comment:

With https://bugs.python.org/issue45963 both frame objects and generators 
contain space for the frame, so no allocation occurs on exit.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue45786>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46374] Assertion failed in ceval.c

2022-01-14 Thread Mark Shannon


Mark Shannon  added the comment:

I can reproduce this on linux.

The problem is that when closing a generator that has not started, the frame 
may not be in a valid state (hence the assertion failure).

This should be fixed by implementing 
https://github.com/faster-cpython/ideas/discussions/217 as that will ensure 
that the frame is in a valid state before the generator is created.

I'll make implementing https://github.com/faster-cpython/ideas/discussions/217 
a priority in order to fix this.

--
assignee:  -> Mark.Shannon
nosy: +brandtbucher

___
Python tracker 
<https://bugs.python.org/issue46374>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46331] 3.11: tracing revisits class line after class docstring

2022-01-14 Thread Mark Shannon


Change by Mark Shannon :


--
assignee:  -> Mark.Shannon
keywords: +patch
stage:  -> patch review

___
Python tracker 
<https://bugs.python.org/issue46331>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46389] 3.11: unused generator comprehensions cause f_lineno==None

2022-01-17 Thread Mark Shannon


Mark Shannon  added the comment:

This has the same root cause as https://bugs.python.org/issue46374:
closing the unused generator expression exposes the generator's frame before it 
has been initialized.

--

___
Python tracker 
<https://bugs.python.org/issue46389>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46409] Add a new bytecode instruction to create generators

2022-01-17 Thread Mark Shannon


New submission from Mark Shannon :

(including coroutines, and async generators)

We now make the start and resumption of Python functions explicit in the 
bytecode which allows us to initialize frames in the bytecode without exposing 
incomplete frames tracing, the GC etc.
However, we still expose incomplete frames when creating generators.

By making the creation of generators explicit in the bytecode we first create 
the frame, then the generator and all is well.

See https://bugs.python.org/issue46374 and https://bugs.python.org/issue46389 
for examples.

--
assignee: Mark.Shannon
components: Interpreter Core
messages: 410764
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Add a new bytecode instruction to create generators
type: behavior
versions: Python 3.11

___
Python tracker 
<https://bugs.python.org/issue46409>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46409] Add a new bytecode instruction to create generators

2022-01-17 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +28836
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30633

___
Python tracker 
<https://bugs.python.org/issue46409>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46389] 3.11: unused generator comprehensions cause f_lineno==None

2022-01-17 Thread Mark Shannon


Change by Mark Shannon :


--
assignee:  -> Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46389>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46374] Assertion failed in ceval.c

2022-01-17 Thread Mark Shannon


Change by Mark Shannon :


--
priority: normal -> release blocker

___
Python tracker 
<https://bugs.python.org/issue46374>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46405] Warning compiling main on Windows

2022-01-17 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset a4bc2218d270c4c7a898c8b3967c8c271afe9abe by Kumar Aditya in 
branch 'main':
bpo-46405: fix msvc compiler warnings (GH-30627)
https://github.com/python/cpython/commit/a4bc2218d270c4c7a898c8b3967c8c271afe9abe


--

___
Python tracker 
<https://bugs.python.org/issue46405>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46161] `class A(1, 2, 3, **d): pass` gives bad bytecode

2022-01-17 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset c118c2455c95baea08045dc64963600b7a56b6fd by zq1997 in branch 
'main':
bpo-46161: Fix bug in starunpack_helper in compile.c (GH-30235)
https://github.com/python/cpython/commit/c118c2455c95baea08045dc64963600b7a56b6fd


--

___
Python tracker 
<https://bugs.python.org/issue46161>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46409] Add a new bytecode instruction to create generators

2022-01-20 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset b04dfbbe4bd7071d46c8688c2263726ea31d33cd by Mark Shannon in 
branch 'main':
bpo-46409: Make generators in bytecode (GH-30633)
https://github.com/python/cpython/commit/b04dfbbe4bd7071d46c8688c2263726ea31d33cd


--

___
Python tracker 
<https://bugs.python.org/issue46409>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46409] Add a new bytecode instruction to create generators

2022-01-20 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue46409>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40116] Regression in memory use of shared key dictionaries for "compact dicts"

2022-01-20 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue40116>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45947] Place dict (and values) pointers at a fixed (negative) offset from the base of the object.

2022-01-20 Thread Mark Shannon


Change by Mark Shannon :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue45947>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43683] Handle generator (and coroutine) state in the bytecode.

2022-01-20 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +28914
pull_request: https://github.com/python/cpython/pull/30723

___
Python tracker 
<https://bugs.python.org/issue43683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42197] Disable automatic update of frame locals during tracing

2022-01-21 Thread Mark Shannon


Mark Shannon  added the comment:

While the various frame and debugger PEPs that are open offer a better solution 
to this, they might not be accepted for 3.11.

So I'd like to revisit this.


Removing the calls to `PyFrame_FastToLocals` and friends cuts the overhead of 
tracing down a lot. A *lot*.

Using Fabio's example I get the following slowdowns:
No calls to   PyFrame_FastToLocals`: x10
Main branch with one local variable: x16
Main branch with six locals variables: x50

This is quite a compelling reason to remove the calls to `PyFrame_FastToLocals`.

The questions is "what will we break doing this"?
All the tests pass, so that seems promising.

Ned, how would this impact coverage.py?

--
nosy: +nedbat

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42197] Disable automatic update of frame locals during tracing

2022-01-21 Thread Mark Shannon


Mark Shannon  added the comment:

Yes the PR has all the changes.

It is just the changes sysmodule.c that you need:
https://github.com/python/cpython/pull/23028/files#diff-a3a5c73931235f7f344c072dc755d6508e13923db3f5d581c5e88652075871cb

--

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue42197] Disable automatic update of frame locals during tracing

2022-01-21 Thread Mark Shannon


Mark Shannon  added the comment:

Or you can use this branch:
https://github.com/faster-cpython/cpython/tree/dont-fast-to-locals-in-trampoline

--

___
Python tracker 
<https://bugs.python.org/issue42197>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46465] Regression caused by CALL_FUNCTION specialization for C function calls

2022-01-23 Thread Mark Shannon


Mark Shannon  added the comment:

Is this a bug?

Signal handling in Python is asynchronous. 
https://docs.python.org/3/library/signal.html#execution-of-python-signal-handlers

The example code tests whether the interpreter responds synchronously and 
immediately.

If you add `for _in range(1): pass` or a call to any Python function in between 
the `os.kill(pid, SIGNUM)` and the `if not self.called:` then the test passes.

--

___
Python tracker 
<https://bugs.python.org/issue46465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46465] Regression caused by CALL_FUNCTION specialization for C function calls

2022-01-24 Thread Mark Shannon


Mark Shannon  added the comment:

> So it's also a threads scheduling issue, no?

No, it isn't. The interpreter checks the eval breaker frequently enough. It 
checks on all back edges and on all calls to Python functions.

The test probably needs to be fixed, or extended. It is signals sent from 
another process or thread that we should be testing for.

I'm happy to merge PR 30826, but if you are really concerned about prompt 
delivery of signals, then you should be worried about C extensions.

If you are worried about being able to interrupt programs, then you also
worry about poorly written Python code that uses `except:`.

Regarding C extensions, I think clear documentation that extension authors need 
to check for signals in any code that might run for a few hundred microseconds 
or longer is the best approach.

For `except:`, maybe we could issue a syntax warning, as `except:` is 
universally considered to be bad practice.

--

___
Python tracker 
<https://bugs.python.org/issue46465>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43683] Handle generator (and coroutine) state in the bytecode.

2022-01-24 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 0367a36fdc36b9c909c4d5acf7cde6ceeec0ba69 by Mark Shannon in 
branch 'main':
bpo-43683: Streamline YIELD_VALUE and SEND (GH-30723)
https://github.com/python/cpython/commit/0367a36fdc36b9c909c4d5acf7cde6ceeec0ba69


--

___
Python tracker 
<https://bugs.python.org/issue43683>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46329] Split up the CALL_NO_KW and CALL_KW instructions.

2022-01-24 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +29036
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30855

___
Python tracker 
<https://bugs.python.org/issue46329>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46420] Use NOTRACE_DISPATCH in specialized opcodes

2022-01-25 Thread Mark Shannon


New submission from Mark Shannon :


New changeset 96bf84d57a7c29544866a6c20231603049de4919 by Dennis Sweeney in 
branch 'main':
bpo-46420: Use NOTRACE_DISPATCH() in specialized opcodes (GH-30652)
https://github.com/python/cpython/commit/96bf84d57a7c29544866a6c20231603049de4919


--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46420>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46532] Improve effeciency of PRECALL/CALL instructions

2022-01-26 Thread Mark Shannon


New submission from Mark Shannon :

The PRECALL/CALL bytecode pair for calls is new (still in review at time of 
writing) and is not as efficient as it could be.

Some possible improvements are:

Transfer refcount of func when making a frame.

NULL call_shape.kwnames after use instead of in PRECALL, as many specialized 
instructions already assert that it is NULL.

Specialize the PRECALL instructions. Either for simple cases like `type(arg)` 
so we can skip the CALL, or for complex cases like `PythonClass(args)` where 
PRECALL can create object, and set up the frame so that `CALL` calls the 
__init__ function.

--
assignee: Mark.Shannon
messages: 411721
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Improve effeciency of PRECALL/CALL instructions

___
Python tracker 
<https://bugs.python.org/issue46532>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46532] Improve effeciency of PRECALL/CALL instructions

2022-01-26 Thread Mark Shannon


Mark Shannon  added the comment:

Possibly consider replacing the specializations for `str(arg)` and `tuple(arg)` 
with a more general bytecode that can be used for other objects as well.

--

___
Python tracker 
<https://bugs.python.org/issue46532>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46533] Specialize for staticmethods and classmethods

2022-01-26 Thread Mark Shannon


New submission from Mark Shannon :

Calls of the form `x.m(args)` where either `x = X` or x = X()` and X is a 
class, and `m` is a classmethod or staticmethod are not currently specialized.

Typically the `x.m()` will translate to:

LOAD_FAST x
LOAD_METHOD "m"
PRECALL_METHOD 0
CALL 0

Since classmethods and staticmethods are descriptors, only the LOAD_METHOD 
should need specializing. The PRECALL/CALL will be able to handle the resulting 
values without modification.

--
messages: 411725
nosy: Mark.Shannon, kj
priority: normal
severity: normal
status: open
title: Specialize for staticmethods and classmethods

___
Python tracker 
<https://bugs.python.org/issue46533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46533] Specialize for staticmethods and classmethods

2022-01-26 Thread Mark Shannon


Mark Shannon  added the comment:

For classmethods, I expect the savings to come from not creating a bound-method 
and from better specialization of the following call.

classmethod case:

>>> class C:
... @classmethod
... def m(*args):
...  pass
... 
>>> C.m
>
>>> C().m
>

So, without specialization LOAD_METHOD "m" has the following stack effect (with 
`x = C()`):
x -> NULL 
C -> NULL 
With specialization:
x -> C 
C -> C 

For static methods the saving is less as there is no change in stack effect, 
but we do save the lookup, and we can reuse existing bytecodes.


Neither classmethod or staticmethod should have Py_TPFLAGS_METHOD_DESCRIPTOR 
set, as they have different semantics.

--

___
Python tracker 
<https://bugs.python.org/issue46533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46166] Get "self" args or non-null co_varnames from frame object with C-API

2022-01-27 Thread Mark Shannon


Mark Shannon  added the comment:

Yes, we should expose the tuple of variable names, both in Python and in the 
C-API.

Would something like
`PyCodeObject_GetVariableName()` and `PyCodeObject_GetVariableKind()` work?


In the meantime, since you were reading `co_varnames` directly, why not read 
`co_localsplusnames` directly? 


OOI, how do you cope with non-local self?

--

___
Python tracker 
<https://bugs.python.org/issue46166>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46476] Not all memory allocated by _Py_Quicken() is released at Python exit

2022-01-27 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 26b0482393a313e3bda364a35e7417e9db52c1c4 by Christian Heimes in 
branch 'main':
bpo-46476: Simplify and fix _PyStaticCode_Dealloc (GH-30965)
https://github.com/python/cpython/commit/26b0482393a313e3bda364a35e7417e9db52c1c4


--

___
Python tracker 
<https://bugs.python.org/issue46476>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46528] Simplify the VM's stack manipulations

2022-01-27 Thread Mark Shannon


Mark Shannon  added the comment:

Timings for individual instructions are a bit meaningless, as out-of-order 
execution and speculation on modern CPUs makes it hard to pin down the timing 
of anything.

I did an experiment to double the number of instructions. It slowed things down 
by ~10%, so increasing the number of instructions by 1% would be expected to 
result in a slowdown of 0.1%.

In other words, this is going to make little or no difference to performance. 
It does make things cleaner and simpler though, which has its own benefits.

--

___
Python tracker 
<https://bugs.python.org/issue46528>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46524] test_peg_generator takes 8 minutes on Windows

2022-01-28 Thread Mark Shannon


Mark Shannon  added the comment:

It's plenty slow on linux as well.

I like the idea of starting the slower tests first.
The long tail of slow tests is annoying when running `make -j12 test`.

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46524>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46329] Split up the CALL_NO_KW and CALL_KW instructions.

2022-01-28 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 89fd7c34520aac493a8784a221366ed04452612b by Mark Shannon in 
branch 'main':
bpo-46329: Split calls into precall and call instructions. (GH-30855)
https://github.com/python/cpython/commit/89fd7c34520aac493a8784a221366ed04452612b


--

___
Python tracker 
<https://bugs.python.org/issue46329>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-01-28 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +29168
pull_request: https://github.com/python/cpython/pull/30989

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46533] Specialize for staticmethods and classmethods

2022-01-28 Thread Mark Shannon


Change by Mark Shannon :


--
keywords: +patch
pull_requests: +29169
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30990

___
Python tracker 
<https://bugs.python.org/issue46533>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-01-28 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 90ab138bbdc63763ad825ed6d4821367c09c4015 by Mark Shannon in 
branch 'main':
bpo-46072: Add simple stats for Python calls. (GH-30989)
https://github.com/python/cpython/commit/90ab138bbdc63763ad825ed6d4821367c09c4015


--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue17206] Py_XDECREF() expands its argument multiple times

2022-01-30 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +29209
pull_request: https://github.com/python/cpython/pull/30855

___
Python tracker 
<https://bugs.python.org/issue17206>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46329] Split up the CALL_NO_KW and CALL_KW instructions.

2022-01-31 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +29216
pull_request: https://github.com/python/cpython/pull/31033

___
Python tracker 
<https://bugs.python.org/issue46329>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46451] Tracing causes module globals to be mutated when calling functions from C

2022-02-01 Thread Mark Shannon


Mark Shannon  added the comment:

Can you reproduce this in pure Python?
If not, can you produce a minimal reproducer using just NumPy?

If you can't do either, I'm going to have to assume that this is a NumPy or 
Pandas bug.

Maybe NumPy or Pandas is accessing CPython internals, but not via the C-API, 
and those internals changed between 3.9 and 3.10?

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46451>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46561] Descriptor resolution should own arguments passed to descriptors

2022-02-01 Thread Mark Shannon


Mark Shannon  added the comment:

# This needs to be C code for this to fail, I'm writing it in Python for 
clarity and brevity

class D:
def __get__(self, instance, owner):
   del C.d
   # There are now no strong references to self
   self.whatever # Access to freed memory

# This can be Python

class C:
d = D()

C.d

--
nosy: +Mark.Shannon

___
Python tracker 
<https://bugs.python.org/issue46561>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-02-01 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +29233
pull_request: https://github.com/python/cpython/pull/31051

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-02-01 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 48be46ec1f3f8010570165daa1da4bf9961f3a83 by Mark Shannon in 
branch 'main':
bpo-46072: Add some object layout and allocation stats (GH-31051)
https://github.com/python/cpython/commit/48be46ec1f3f8010570165daa1da4bf9961f3a83


--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-02-01 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +29242
pull_request: https://github.com/python/cpython/pull/31060

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-02-02 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 187930f74c44e460ba09c60ba5d9bb4fac543d8f by Mark Shannon in 
branch 'main':
bpo-46072: Add some frame stats. (GH-31060)
https://github.com/python/cpython/commit/187930f74c44e460ba09c60ba5d9bb4fac543d8f


--

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue46072] Unify handling of stats in the CPython VM

2022-02-02 Thread Mark Shannon


Change by Mark Shannon :


--
pull_requests: +29263
pull_request: https://github.com/python/cpython/pull/31079

___
Python tracker 
<https://bugs.python.org/issue46072>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45773] Compiler hangs during jump elimination

2022-02-03 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset e0433c1e70254d4d0357a9e14596929a04bdf769 by Brandt Bucher in 
branch 'main':
bpo-45773: Remove invalid peephole optimizations (GH-31066)
https://github.com/python/cpython/commit/e0433c1e70254d4d0357a9e14596929a04bdf769


--

___
Python tracker 
<https://bugs.python.org/issue45773>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue45885] Specialize COMPARE_OP

2022-02-03 Thread Mark Shannon


Mark Shannon  added the comment:


New changeset 674ab66ebdf06f187e193a3d7bde13b71ba0f9af by Dennis Sweeney in 
branch 'main':
bpo-45885: Add more stats for COMPARE_OP in specialize.c (GH-31040)
https://github.com/python/cpython/commit/674ab66ebdf06f187e193a3d7bde13b71ba0f9af


--

___
Python tracker 
<https://bugs.python.org/issue45885>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   3   4   5   6   7   8   9   10   >