Eric Snow added the comment:
On Wed, Oct 20, 2021 at 8:38 AM Marc-Andre Lemburg
wrote:
> Yes, 3.10 is next, once I have 3.9 ironed out. And then I'll give 3.11 a try.
Thanks!
> Sorry, I wasn't clear. PGO is not a problem for test_embed. I just
> wonder why the test_embe
Eric Snow added the comment:
On Wed, Oct 20, 2021 at 11:11 AM Barry A. Warsaw wrote:
> Maybe we should relax the restriction on module.__spec__.parent so that it's
> writeable just like module.__package__.
Hmm, I hadn't realized __package__ is used in __import__(). T
Eric Snow added the comment:
On Mon, Jan 7, 2019 at 11:41 PM Eric V. Smith wrote:
> Namespace packages (PEP 420) predate ModuleSpec (PEP 451). So, I think this
> probably happened when 451 was implemented. Maybe Eric Snow recalls?
PEP 451 talks about this a little
(https://www.pyth
Eric Snow added the comment:
On Wed, Oct 20, 2021 at 6:11 PM Barry A. Warsaw wrote:
> I guess a question to answer then is whether we philosophically want the
> module attributes to be equivalent to the spec attributes. And by
> equivalent, I mean enforced to be exactly so, and thu
Eric Snow added the comment:
On Wed, Oct 20, 2021 at 6:11 PM Barry A. Warsaw wrote:
> This is what leads me to think that having a proxy to keep them in sync and
> relaxing the read-only restriction is the path forward, even if writing
> __package__ doesn’t make sense. It also s
Eric Snow added the comment:
I ended up writing up more thoughts on this in the issue about __package__:
https://bugs.python.org/issue45540#msg404619.
Users can call importlib.util.find_spec(), which is probably good enough to
figure out how a module was originally imported. So I don&#
Eric Snow added the comment:
On Thu, Oct 21, 2021 at 11:54 AM Barry A. Warsaw wrote:
> Thanks for your comprehensive comments Eric! While I digest them, my TL;DR
> is that we probably need a PEP to describe everything from the current
> situation, to the the desired end state and
Eric Snow added the comment:
See bpo-35673. Also, it is actually in importlib._bootstrap_external.
--
nosy: +eric.snow
___
Python tracker
<https://bugs.python.org/issue18
Eric Snow added the comment:
New changeset 17c61045c51512add61a9e75e9c7343cf4e4fb82 by Eric Snow in branch
'main':
bpo-45506: Normalize _PyPathConfig.stdlib_dir when calculated. (#29040)
https://github.com/python/cpython/commit/17c61045c51512add61a9e75e9c734
Eric Snow added the comment:
Thanks for bringing this up, Greg!
--
___
Python tracker
<https://bugs.python.org/issue45506>
___
___
Python-bugs-list mailin
Eric Snow added the comment:
New changeset 6afb285ff0790471a6858e44f85d143f07fda70c by Eric Snow in branch
'main':
bpo-45020: Add tests for the -X "frozen_modules" option. (gh-28997)
https://github.com/python/cpython/commit/6afb285ff0790471a685
Change by Eric Snow :
--
keywords: +patch
pull_requests: +27485
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29222
___
Python tracker
<https://bugs.python.org/issu
Eric Snow added the comment:
FYI, I figured out the problem on my end. I wasn't using an installed python.
Once I did it worked fine.
--
___
Python tracker
<https://bugs.python.org/is
New submission from Eric Snow :
I found that Tools/freeze doesn't get tested. It has a "test" directory but
it's more of a rudimentary sanity check that must be run manually. Since the
tool isn't tested through our test suite, there's a high likelihood it
Change by Eric Snow :
--
keywords: +patch
pull_requests: +27506
stage: test needed -> patch review
pull_request: https://github.com/python/cpython/pull/29222
___
Python tracker
<https://bugs.python.org/issu
Change by Eric Snow :
--
pull_requests: -27485
___
Python tracker
<https://bugs.python.org/issue45096>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Eric Snow :
--
stage: patch review -> needs patch
___
Python tracker
<https://bugs.python.org/issue45096>
___
___
Python-bugs-list mailing list
Un
Eric Snow added the comment:
New changeset 13d9205f4057f80a25d410ad123876dc60cd by Eric Snow in branch
'main':
bpo-45629: Add a test for the "freeze" tool. (gh-29222)
https://github.com/python/cpython/commit/13d9205f4057f80a
Change by Eric Snow :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
New submission from Eric Snow :
(forked from https://bugs.python.org/issue45020#msg404344)
On Mon, Oct 18, 2021 at 7:14 PM Guido van Rossum wrote:
> I built on Windows with default options (PCbuild\build.bat) and it looks like
> the frozen modules are used by default even though I am r
Eric Snow added the comment:
On Mon, Oct 18, 2021 at 7:14 PM Guido van Rossum wrote:
> Is gh-28940 only for UNIX?
> I built on Windows with default options (PCbuild\build.bat) and it looks like
> the frozen modules are used by default even though I am running in the source
> d
Change by Eric Snow :
--
title: -X frozen_modules not defaulting to "on" on Windows? -> -X
frozen_modules not defaulting to "off" on Windows when running in source tree?
___
Python tracker
<https://
New submission from Eric Snow :
Frozen modules currently have co_filename set to "". It would be
good to have it match __file__ instead. This will require require also fixing
any code objects in the module's co_consts (and so on). It may make sense to
take care of the whi
New submission from Eric Snow :
Currently we freeze all the modules imported during runtime initialization,
except for the encodings module. It has a lot of submodules and this results
in a lot of extra noise in builds. We hadn't frozen it yet because we were
still ironing out ch
New submission from Eric Snow :
Currently we are freezing the modules that are imported during startup.
However, if someone uses "python -m module" then the runtime imports the runpy
module and uses it. Given that this case is quite common, runpy and its
dependencies should
Eric Snow added the comment:
Note that once the get_source() implementation is done, the only missing method
for importlib.abc.InspectLoader is source_to_code(). (See
https://docs.python.org/3/library/importlib.html#importlib.abc.InspectLoader
New submission from Eric Snow :
FrozenImporter (in Lib/importlib/_bootstrap.py) has a get_source() method but
it always returns None. Now that __file__ is set on frozen (stdlib) modules,
we can implement FrozenImporter.get_source() appropriately.
--
components: Library (Lib
New submission from Eric Snow :
Now that __file__ is set on frozen (stdlib) modules, we can add an
implementation for FrozenImporter.get_filename(). This would make
FrozenImporter and implementation of ExecutionLoader. [1]
There is a caveat: in spec_from_loader() [2] we infer the presence
Eric Snow added the comment:
Note that there isn't any intent currently to implement the FileLoader ABC [1],
which would require having FrozenImporter instances, adding the "name" and
"path" attributes to them, as well as the get_data() method. The same goes for
Eric Snow added the comment:
I've opened the following issues related to frozen stdlib modules:
https://bugs.python.org/issue45652
https://bugs.python.org/issue45658
https://bugs.python.org/issue45659
Again, I'm leaving this issue open to deal with the broader question of froze
New submission from Eric Snow :
(See https://bugs.python.org/issue45020#msg402116.)
On Fri, Sep 17, 2021 at 7:56 PM Raymond Hettinger
wrote:
> It would be nice to freeze argparse.py and its dependencies. For
> command-line tools, startup time is important.
This would include freezi
Eric Snow added the comment:
(See https://bugs.python.org/issue45020#msg402118.)
--
___
Python tracker
<https://bugs.python.org/issue45654>
___
___
Python-bug
New submission from Eric Snow :
We're already freezing modules needed for bootstrapping the runtime, as well as
the modules imported during startup. [1][2] We're also planning on freezing
argparse. [3] There may be other modules (or files) worth freezing (assuming
we don't
Eric Snow added the comment:
One to consider that isn't necessarily used that commonly is sysconfig.
However, that could also involve freezing the "sysconfigdata" file (without
needing to expose it as a module).
--
___
Python
Eric Snow added the comment:
I consider this done. There is some lingering follow-up work, for which I've
created a number of issues:
* https://bugs.python.org/issue45396 - -X frozen_modules=off ignores custom
frozen modules
* https://bugs.python.org/issue45395 - custom modules ove
Eric Snow added the comment:
On Thu, Oct 28, 2021 at 12:15 PM Marc-Andre Lemburg
wrote:
> encodings is a package. I think you first have to check whether mixing
> frozen and non-frozen submodules are even supported. I've never tried
> having only part of a package frozen.
It wo
Change by Eric Snow :
--
pull_requests: +27560
pull_request: https://github.com/python/cpython/pull/28778
___
Python tracker
<https://bugs.python.org/issue45
Eric Snow added the comment:
New changeset 074fa5750640a067d9894c69378a00ceecc3b948 by Eric Snow in branch
'main':
bpo-45395: Make custom frozen modules additions instead of replacements.
(gh-28778)
https://github.com/python/cpython/commit/074fa5750640a067d9894c69378a00
Change by Eric Snow :
--
pull_requests: +27564
pull_request: https://github.com/python/cpython/pull/29300
___
Python tracker
<https://bugs.python.org/issue45
Change by Eric Snow :
--
pull_requests: +27565
pull_request: https://github.com/python/cpython/pull/29301
___
Python tracker
<https://bugs.python.org/issue45
Eric Snow added the comment:
The behavior has been addressed in the fix for bpo-45395. However, I'd like to
change the name of the -X option from "frozen_modules" to "frozen_stdlib", to
avoid any confusion in the future. After that this issue is done.
---
Eric Snow added the comment:
This is done now.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Eric Snow :
--
resolution: fixed ->
stage: resolved -> patch review
status: closed -> open
___
Python tracker
<https://bugs.python.or
New submission from Eric Snow :
Currently we don't test builds done outside the source tree, neither on GitHub
nor the buildbots. [1] As a result, such builds get broken occasionally. I've
certainly broken then a couple times, inadvertently. It would be helpful if we
tested o
Change by Eric Snow :
--
nosy: +eric.snow
___
Python tracker
<https://bugs.python.org/issue45696>
___
___
Python-bugs-list mailing list
Unsubscribe:
Eric Snow added the comment:
New changeset 7b438282d39e742b2628a32e7c6699a140ff4cfb by Eric Snow in branch
'main':
bpo-45506: Stop skipping test_embed. (gh-29300)
https://github.com/python/cpython/commit/7b438282d39e742b2628a32e7c6699
Eric Snow added the comment:
On Wed, Nov 3, 2021 at 1:21 PM Steve Dower wrote:
> Last remaining test failure is one that I can't figure out on my own - the
> freeze test is rerunning a CPython build (on Linux) and is apparently
> building getpath.c with the ".c.o&qu
Eric Snow added the comment:
maybe related:
https://github.com/python/cpython/pull/29041#discussion_r734085599
--
___
Python tracker
<https://bugs.python.org/issue45
Eric Snow added the comment:
On Wed, Nov 3, 2021 at 6:25 PM Steve Dower wrote:
> Now to find out why the old getpath could somehow locate the stdlib but new
> getpath cannot... (I'm guessing it is finding the "original" stdlib rather
> than the fresh clone, since AFA
Eric Snow added the comment:
Thanks for the report, Petr! I'll take a look.
--
assignee: -> eric.snow
components: +Tests
stage: -> needs patch
___
Python tracker
<https://bugs.python.
Change by Eric Snow :
--
keywords: +patch
pull_requests: +2
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29527
___
Python tracker
<https://bugs.python.org/issu
Eric Snow added the comment:
On Thu, Nov 11, 2021 at 6:27 PM Steve Dower wrote:
> rather than streamline anything by changing it (yet). We can do those once we
> know we've got something working.
+1
--
___
Python trac
New submission from Eric Snow :
Currently there are still a bunch of PyObject static variables in the code that
need to become per-interpreter. This includes quite a few static types (e.g.
in Objects/*.c), as well as freelists and caches. To make the transition
easier I'd like to mov
Change by Eric Snow :
--
keywords: +patch
pull_requests: +27975
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/29737
___
Python tracker
<https://bugs.python.org/issu
Eric Snow added the comment:
IIRC, Programs/_freeze_module.c was originally a C implementation of the basic
functionality in Tools/freeze/freeze.py.
--
nosy: +eric.snow
___
Python tracker
<https://bugs.python.org/issue45
Eric Snow added the comment:
Ideally we would not have two scripts that do the same thing.
--
___
Python tracker
<https://bugs.python.org/issue45949>
___
___
New submission from Eric Snow :
Some relatively recent changes broke "./python Tools/c-analyzer/c-analyzer.py
...". Also, Tools/c-analyzer/TODO is a bit out of date.
--
assignee: eric.snow
components: Demos and Tools
messages: 407474
nosy: eric.snow
priority: normal
pul
Eric Snow added the comment:
New changeset ee94aa0850191712e6adfc1f4a9df08ec3240195 by Eric Snow in branch
'main':
bpo-45952: Get the C analyzer tool working again. (gh-29882)
https://github.com/python/cpython/commit/ee94aa0850191712e6adfc1f4a9df0
Change by Eric Snow :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
New submission from Eric Snow :
Currently we allocate each new PyInterpreterState in PyInterpreterState_New().
Furthermore, PyInterpreterState is full of pointers which are almost all
allocated on the heap during runtime init. We can statically allocate (and
initialize?) most of what goes
Change by Eric Snow :
--
keywords: +patch
pull_requests: +28109
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29883
___
Python tracker
<https://bugs.python.org/issu
Eric Snow added the comment:
Hurray! Thanks, Steve!
--
___
Python tracker
<https://bugs.python.org/issue45582>
___
___
Python-bugs-list mailing list
Unsub
Eric Snow added the comment:
The key point is that loaders should be using the spec, not any of the module
attrs (like `__file__` and `__cached__`), nor setting them.
For the specific paragraph I referenced, it would look more like:
It is also appropriate to set __cached__ when __file__
New submission from Eric Snow :
There are a few things in `_PyRuntimeState`, `PyInterpreterState`,
`PyThreadState`, and there initialization (Python/pystate.c &
Python/pylifecycle.c) that would benefit from some minor cleanup. Normally I
wouldn't bother (due to the cost of churn),
Change by Eric Snow :
--
keywords: +patch
pull_requests: +28195
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29970
___
Python tracker
<https://bugs.python.org/issu
Change by Eric Snow :
--
pull_requests: +28196
pull_request: https://github.com/python/cpython/pull/29971
___
Python tracker
<https://bugs.python.org/issue46
Change by Eric Snow :
--
pull_requests: +28197
pull_request: https://github.com/python/cpython/pull/29972
___
Python tracker
<https://bugs.python.org/issue46
Change by Eric Snow :
--
pull_requests: +28198
pull_request: https://github.com/python/cpython/pull/29973
___
Python tracker
<https://bugs.python.org/issue46
Eric Snow added the comment:
New changeset 8262c96bcc1841188866c1b022d9087e89639d98 by Eric Snow in branch
'main':
bpo-46008: Return void from _PyEval_InitState(). (gh-29970)
https://github.com/python/cpython/commit/8262c96bcc1841188866c1b022d908
Eric Snow added the comment:
New changeset 313f92a57bc3887026ec16adb536bb2b7580ce47 by Eric Snow in branch
'main':
bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)
https://github.com/python/cpython/commit/313f92a57bc3887026ec16adb536bb
Eric Snow added the comment:
New changeset 1f384e318481532323bb9076f4447bc02da07209 by Eric Snow in branch
'main':
bpo-46008: Stop calling _PyThreadState_Init() in new_threadstate(). (gh-29973)
https://github.com/python/cpython/commit/1f384e318481532323bb9076f4447b
Change by Eric Snow :
--
pull_requests: +28203
pull_request: https://github.com/python/cpython/pull/29977
___
Python tracker
<https://bugs.python.org/issue46
Change by Eric Snow :
--
pull_requests: +28204
pull_request: https://github.com/python/cpython/pull/29978
___
Python tracker
<https://bugs.python.org/issue46
Eric Snow added the comment:
New changeset 758b74e71eb22e1e83a9eb937d1c015e461745a1 by Eric Snow in branch
'main':
bpo-46008: Add _PyInterpreterState_Main(). (gh-29978)
https://github.com/python/cpython/commit/758b74e71eb22e1e83a9eb937d1c01
Eric Snow added the comment:
New changeset 32a67246b0d1e08cd50fc3bfa58052cfeb515b2e by Eric Snow in branch
'main':
bpo-46008: Move Py*State init into distinct functions. (gh-29977)
https://github.com/python/cpython/commit/32a67246b0d1e08cd50fc3bfa58052
Eric Snow added the comment:
At this point all the changes I was considering have been made, except for
splitting type/object init into logical phases. That may or may not happen.
Regardless, it can be done in a new issue.
--
resolution: -> fixed
stage: patch review -> re
Change by Eric Snow :
--
pull_requests: +28221
pull_request: https://github.com/python/cpython/pull/29998
___
Python tracker
<https://bugs.python.org/issue46
Eric Snow added the comment:
New changeset c8749b578324ad4089c8d014d9136bc42b065343 by Eric Snow in branch
'main':
bpo-46008: Make runtime-global object/type lifecycle functions and state
consistent. (gh-29998)
https://github.com/python/cpyt
Eric Snow added the comment:
Nice work, Phil. We need a backport to 3.8, but the miss-islington bot is
having trouble with it. [1] Either we can try flipping the "needs backport to
3.8" label again or you could create a backport PR yourself (like
miss-islington suggested). If y
New submission from Eric Snow :
The C-API docs are a bit sparse on the interplay between C fork() and the
CPython runtime.
--
assignee: eric.snow
components: Documentation
messages: 356705
nosy: eric.snow, gregory.p.smith, pconnell
priority: normal
pull_requests: 16684
severity
Change by Eric Snow :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Eric Snow added the comment:
Flipping the label one more time did the trick. Thanks again, Phil!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Eric Snow added the comment:
I wouldn't worry about the c-analyzer stuff for now. I plan on re-generating
the file in the near future.
--
nosy: +eric.snow
___
Python tracker
<https://bugs.python.org/is
Eric Snow added the comment:
Thanks so much for getting this done, Victor!
> I'm not fully happy with this solution
Should we have an issue open for finding a better solution? Are there risks
with what you did that we don't w
Eric Snow added the comment:
Did I mention that you're my hero? :)
--
___
Python tracker
<https://bugs.python.org/issue36854>
___
___
Python-bugs-list m
Eric Snow added the comment:
Victor's comment [1] on that related issue, implies that this may no longer be
a problem (on 3.8). Please check. Thanks!
[1] https://bugs.python.org/issue17978#msg355166
--
status: open -> pending
__
Eric Snow added the comment:
tl;dr Py_Finalize() probably *should* only be allowed under the main
interpreter.
As you know well, when the runtime starts we do it at first relative to a
partially initialized main interpreter and the finish initialization with a
fully operational main
Eric Snow added the comment:
> * Is Python supposed to magically destroy the 3 interpreters?
Doesn't it? Gah. I guess I was thinking of PyOS_AfterFork_Child(), which
calls _PyInterpreterState_DeleteExceptMain(). :/ In September there was a nice
comment right above Py_FinalizeEx(
Eric Snow added the comment:
I could swear the topic of destroy-everything-in-PyFinalize has come up before
but I don't remember anything specific. Doing so there sure makes sense
though...
--
___
Python tracker
<https://bugs.py
Change by Eric Snow :
--
keywords: +patch
pull_requests: +16843
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/16293
___
Python tracker
<https://bugs.python.org/issu
Eric Snow added the comment:
Sorry I haven't gotten back to you sooner, Kyle. Thanks for working on this.
I'm looking at your PR right now.
--
___
Python tracker
<https://bugs.python.o
Eric Snow added the comment:
BTW, Kyle, your problem-solving approach on this is on-track. Don't get
discouraged. This stuff is tricky. :)
--
___
Python tracker
<https://bugs.python.org/is
Eric Snow added the comment:
Thus far these are the failures we've seen:
* not running when we expect it to be running:
* interpreters.is_running(interp)
* interpreters.run_string(interp, ...)
* interpreters.destroy(interp)
* can't find the interpreter even though we ex
Eric Snow added the comment:
Thanks, Vinay!
--
___
Python tracker
<https://bugs.python.org/issue36876>
___
___
Python-bugs-list mailing list
Unsubscribe:
Eric Snow added the comment:
FYI, others have been tackling this in separate issues (e.g. Victor, anyone
relative to PEP 384).
--
___
Python tracker
<https://bugs.python.org/issue36
Eric Snow added the comment:
And I *am* still working on the c-analyzer + a test to that runs it, so we can
keep from adding more globals. :)
--
___
Python tracker
<https://bugs.python.org/issue36
Eric Snow added the comment:
Yeah, the case of pickle + __main__ is an awkward one. [1] However, the
approach taken by multiprocessing isn't the right one for subinterpreters.
Multiprocessing operates under 2 design points that do not apply to
subinterpreters:
* every process is ru
Eric Snow added the comment:
In the meantime that leaves the workarounds that @crusaderky originally
identified. You could also:
* manually run __main__ in the subinterpreter first (sort of like
multiprocessing does automatically); this works because the namespace of
__main__ is not reset
Eric Snow added the comment:
FWIW, I have some feedback on the PEP. (See msg357448.) Can we discuss here
or should I open a mailing list thread?
--
nosy: +eric.snow
___
Python tracker
<https://bugs.python.org/issue36
New submission from Eric Snow :
The docs page for the inspect module has a large table describing the special
attributes of various important types. One entry for function attributes is
missing: __module__. It should be added.
Note that __module__ *is* included in the function attributes
501 - 600 of 2629 matches
Mail list logo
