Brett Cannon added the comment:
OK, so you say "to the object found in sys.modules."
--
___
Python tracker
<http://bugs.python.org/issue14628>
___
___
Brett Cannon added the comment:
So importlib just increased the window of vulnerability for this kind of thing.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
Committed in http://hg.python.org/cpython/rev/a281a6622714 (I think our issue
detection algorithm grabs the last issue in a commit message instead of the
first one).
--
assignee: -> brett.cannon
resolution: -> fixed
stage: -> committed
Brett Cannon added the comment:
http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/98/steps/test/logs/stdio
suggests this fix worked.
--
assignee: -> brett.cannon
resolution: -> fixed
stage: needs patch -> committed/rejected
status: open
Changes by Brett Cannon :
--
title: Windows test_import failure -> Windows test_import failure thanks to
ImportError.path
___
Python tracker
<http://bugs.python.org/issu
Changes by Brett Cannon :
--
assignee: -> brett.cannon
resolution: -> fixed
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python
Changes by Brett Cannon :
--
assignee: -> brett.cannon
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.or
Changes by Brett Cannon :
--
assignee: docs@python -> brett.cannon
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python
New submission from Brett Cannon :
See
http://www.python.org/dev/buildbot/all/builders/x86%20XP-5%203.x/builds/104/steps/test/logs/stdio
for the failure, but basically:
==
ERROR: test_UNC_path (test.test_import.PathsTests
Brett Cannon added the comment:
OK, so a cursory look at importlib suggests that the possible costs of those
stat calls (by looking at what has to examine the filesystem) are:
* os.listdir() for caching
* os.path.isdir() for directories if they are a package
* os.path.isfile() for __init__.py
Brett Cannon added the comment:
Time for a recap!
I am personally not bothering with moving imp.get_tag() and imp.get_magic() to
importlib._bootstrap as the amount of C code required to support the public C
API is not worth it. If someone else once to do the work then by all means
attach a
Brett Cannon added the comment:
Thanks for catching that, Vinay! Just waiting for the buildbots to verify my
fix (which also included invalidating the cache which is why it was sporadic
plus cleaning up sys.path properly).
--
___
Python tracker
Brett Cannon added the comment:
Thanks for catching that, Vinay! Buildbots say it worked.
--
resolution: -> fixed
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
New submission from Brett Cannon :
A shell or Python script should be used to verify that importlib.h actually
needs to be rebuilt, and if so make sure that it's even possible (since it
depends on a locally built ./python).
--
components: Build
messages: 158944
nosy: brett.c
Brett Cannon added the comment:
The thing should also do a sanity check on the syntax so as to minimize the
chances of freezing code that has no chance of working.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
To note:
_frozen_importlib is a CPython implementation detail
--
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Brett Cannon added the comment:
On Sun, Apr 22, 2012 at 12:43, Martin v. Löwis wrote:
>
> Martin v. Löwis added the comment:
>
> What do you mean by "sanity check on the syntax"? The current Makefile
> rule to build importlib.h already checks the syntax of _bootstrap
Brett Cannon added the comment:
I don't quite follow. I have no issue with a shell script verifying things by
doing an `hg status` to verify certain truths (or something). I said a shell
*or* Python script to begin with.
--
___
Python tr
New submission from Brett Cannon :
As discussed and agreed to on python-dev, it makes sense to require loaders to
set __loader__ and __package__ so that they can be relied upon by globally
executed code in a module.
The following needs to happen to close this bug:
* Update PEP 302 to say
Changes by Brett Cannon :
--
dependencies: +Make import machinery explicit
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Python-bugs-list mailin
Brett Cannon added the comment:
First off, you should separate the patches for get_magic() and get_tag().
Second, why is there _get_pyc_magic_int() when it is never called? Third, all
of this would be greatly simplified if you just had a _RAW_MAGIC_NUMBER of
3220, did the bytes object
Brett Cannon added the comment:
No, I don't want a shell script just to have one. =)
The status quo seems to work, but people like Georg think it's partially luck
that it does and if hg changes its semantics that will cause
New submission from Brett Cannon :
> ./python.exe -c "import urllib.parse as x; import imp; imp.reload(x)"
Assertion failed: (gc->gc.gc_refs != 0), function visit_decref, file
Modules/gcmodule.c, line 327.
zsh: abort ./python.exe -c "import urllib.parse as x; import im
Brett Cannon added the comment:
Segfault:
> ./python.exe -c "import importlib.abc as x; import imp; imp.reload(x)"
>
Traceback (most recent call last):
File "", line 1, in
File "", line 611, in
Brett Cannon added the comment:
Attached is my (failing) attempt at making import explicit. Unfortunately I
have four failing tests, 3 of which revolve around __main__ (which is why I
added Nick to see if he had any ideas):
test_cmd_line_script test_runpy test_threaded_import
Changes by Brett Cannon :
--
keywords: +patch
Added file: http://bugs.python.org/file25307/explicit.diff
___
Python tracker
<http://bugs.python.org/issue14
Changes by Brett Cannon :
--
nosy: +ncoghlan
___
Python tracker
<http://bugs.python.org/issue14605>
___
___
Python-bugs-list mailing list
Unsubscribe:
Brett Cannon added the comment:
Loosening the type restrictions is fine.
As for testing, that's what the test suite is supposed to do. So if need be
just write tests in Python that exercise the C code.
--
___
Python tracker
Brett Cannon added the comment:
To try and narrow down the issue, I now have separate patches for an explicit
sys.meta_path and a sys.path_hooks. Both fail on tests, but for different
reasons.
The explicit sys.meta_path is still failing on the __main__ issue on three of
the tests and for
Changes by Brett Cannon :
Added file: http://bugs.python.org/file25309/explicit_path_hooks.diff
___
Python tracker
<http://bugs.python.org/issue13959>
___
___
Python-bug
Brett Cannon added the comment:
So why the mutation? Are you that worried someone is going to import
importlib._bootstrap directly?
This also costs in development complexity because not only do you have to run
'make' to get changes to be testable, but it also leads to difficult
Brett Cannon added the comment:
Updated patch for an explicit sys.path_hooks that simply tweaks the
test_cmd_line_script tests to stop requiring an absolute path on the files and
instead verifies that the relative paths are legit. It also adds warnings when
None is found in
Brett Cannon added the comment:
Issue #14657 is tracking the discussion of
_frozen_importlib/importlib._bootstrap.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
I should also mention that all of this becomes much less important once issue
#14605 is finished because at that point sys.meta_path and sys.path_hooks will
have _frozen_importlib objects and that will be what importlib works off of
directly. But I still
Brett Cannon added the comment:
Of course you did because you just like making my life a living hell when it
comes to the __file__ attribute. =)
OK, I will have another look when I get home, but last time I dealt with this
the issue is some tests expect a relative path while others expect an
Brett Cannon added the comment:
Loaders are not meant to have a find_module method; that is purely for finders
which can be distinct objects. So having a namespace loader is expected and
there is no expectation that it have a find_module method (actually almost all
of the loaders in
Brett Cannon added the comment:
To start, I'm *not* going to make the final call on this issue's solution. I'm
inches away from importlib burnout and general integration frustration with
trying to clean up the implicit behaviour. So to prevent me from making a bad
decision
Brett Cannon added the comment:
That initial comment is out-of-date. If you look that the commit I made I
documented importlib.machinery._SourcelessFileLoader. I am continuing the
discouragement of using bytecode files as an obfuscation technique (because
it's a bad one), but I decid
Brett Cannon added the comment:
Yeah, having to pass in the name to load_module is silly. I'm seriously
considering making it optional for some loaders when the name was passed in to
the constructor.
One thing I would like to see is that PathFinder take a new, keyword-only
argume
Brett Cannon added the comment:
I documented it explicitly so people can use it if they so choose (e.g. look at
sys._getframe()). If you want to change this that's fine, but I am personally
not going to put the effort in to rename the class, update the tests, and
change the docs for thi
Brett Cannon added the comment:
I don't quite follow what you are suggesting, MAL. Are you saying to freeze
importlib.__init__ and importlib._bootstrap and somehow have improtlib.__init__
choose what to load, frozen or source?
--
___
P
Brett Cannon added the comment:
So basically if you are running in a checkout, grab the source file and compile
it manually since its location is essentially hard-coded and thus you don't
need to care about sys.path and all the other stuff required to do an import,
while using the f
Brett Cannon added the comment:
Modules/getpath.c seems to be where the C code does it when getting paths for
sys.path. So it would be possible to use that same algorithm to set some sys
attribute (e.g. in_checkout or something) much like sys.gettotalrefcount is
optional and only shown when
Brett Cannon added the comment:
That's why I was thinking of tying into Modules/getpath.c because I assume that
would work cross-platform. Is that incorrect?
--
___
Python tracker
<http://bugs.python.org/is
Brett Cannon added the comment:
You can see a little discussion in http://bugs.python.org/issue14642, but it
has been discussed elsewhere and the automatic rebuilding was preferred (but it
is not a requirement to build as importlib.h is in hg
Brett Cannon added the comment:
That solves the "I'm in a checkout" problem but it doesn't tell you necessarily
where the Lib directory is if you e.g. build from within another directory like
Python/, which places the executable and pybuilddir.txt in the current
directory
Brett Cannon added the comment:
I found out why runpy was giving back an absolute file path for __file__;
because pkgutil was doing that through its ImpLoader, unlike what import does
by default which is just taking what path it has and appending file names (and
thus not making anything
Changes by Brett Cannon :
--
assignee: brett.cannon -> ncoghlan
stage: -> patch review
___
Python tracker
<http://bugs.python.org/issue14605>
___
___
Pyth
Brett Cannon added the comment:
What do you mean the loader is only needed to set __loader__? You need the
loader to create the module (or find it in sys.modules to reload), and set all
the attributes properly. If you do this then reloading namespace modules will
become a special case
Brett Cannon added the comment:
To answer MAL's question about startup, I benchmarked on my machine using the
normal_startup benchmark from hg.python.org/benchmarks and the bootstrap work
only caused a 5-6% slowdown in a non-debug build. If you do it in a debug build
it's much wors
Brett Cannon added the comment:
The joys of trying to shoehorn into an existing API. I mean short of adding a
new sys.namespace_loader instead of an explicit keyword argument to FileFinder
I can't think of a better solution.
--
___
Python tr
Brett Cannon added the comment:
This is where a script could help with printing out a warning if the built
Python interpreter is not available.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
When you do a fresh checkout, Python/importlib.h comes after
Lib/importlib/_bootstrap.py and Python/freeze_importlib.py in a lexicographical
sort which leads to it have a newer timestamp and thus not triggering a new
build of the file.
This is why Martin
Brett Cannon added the comment:
So how would you tweak the explicit work I'm doing? The code is going to rely
on sys.path_hooks and sys.meta_path being populated. I guess the frozen code
can set up initially, and then importlib simply substitutes out classes from
the frozen module t
Brett Cannon added the comment:
importlib.find_module() (or get_loader() as it would replace
pkgutil.get_loader() as well) is definitely planned. So is
importlib.util.resolve_name() (although maybe that is basic enough to want
top-level
Brett Cannon added the comment:
Just to document why my explicit sys.path_hooks patch didn't quite change the
meaning of None in sys.path_importer_cache, I found a bunch of places in the
stdlib and in Modules/main.c where NullImporter is explicitly expected to be
returned, so I wanted t
New submission from Brett Cannon :
zipimport's finders that get cached in sys.path_importer_cache should probably
be updated to support importlib.invalidate_caches() (else the module should get
re-implemented in pure Python in importlib and then be tossed).
--
components: Library
Brett Cannon added the comment:
So I'm w/ Antoine that a dict would be better so that the other VMs can add
whatever they want into that object (e.g. Jython could specify what JVM they
are running against) without causing confusion as to what some index means to
each VM. The mutability
Brett Cannon added the comment:
OK, so the todo of this issue is now finished. I am just waiting for the
buildbots to come back green before I close this issue fully.
--
stage: commit review -> committed/rejected
status: open -> pending
___
Changes by Brett Cannon :
--
status: pending -> closed
___
Python tracker
<http://bugs.python.org/issue14605>
___
___
Python-bugs-list mailing list
Unsubscri
Brett Cannon added the comment:
Can this issue be closed, Martin, thanks to your extension?
--
assignee: -> loewis
___
Python tracker
<http://bugs.python.org/issu
Changes by Brett Cannon :
--
resolution: -> fixed
stage: test needed -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Brett Cannon added the comment:
Update time!
With NullImporter dealt with, that leaves get_magic(), get_tag(), reload(), and
get_suffixes() as things to potentially move to Lib/imp.py. I would also like
to re-implement PyImport_ExecCodeModuleObject() as it's keeping a lot of C code
Brett Cannon added the comment:
Fixed by http://hg.python.org/cpython/rev/eb68502731dd
--
assignee: -> brett.cannon
resolution: -> fixed
stage: test needed ->
status: open -> closed
___
Python tracker
<http://bugs.python
Brett Cannon added the comment:
So here is the deal with PyImport_ExecCodeModuleObject(): bootstrapping and
Barry has made this a little annoying. =)
First off, PyImport_ImportFrozenModuleObject() uses
PyImport_ExecCodeModuleObject(), so that precludes just directly importing imp
to handle
Brett Cannon added the comment:
That test is going to stay intermittent until issue #14657 gets resolved else
the exact reason for the failure is going to be hard to debug remotely.
--
___
Python tracker
<http://bugs.python.org/issue13
Brett Cannon added the comment:
So I'm no pkgutil expert, but at least in the case of None in sys.modules, that
triggers at least an ImportError in __import__ if that is come across.
--
___
Python tracker
<http://bugs.python.org/is
Brett Cannon added the comment:
Yes, it could do all of this. One possibility that I was thinking of, was this
could be a setUp()/tearDown() thing as well instead of a context manager.
Another option is a simple decorator. Either way it might be easier to have it
just save state and then in
Brett Cannon added the comment:
I should mention I have a version from importers that is probably out-of-date:
http://code.google.com/p/importers/source/browse/importers/zip.py
--
___
Python tracker
<http://bugs.python.org/issue14
Changes by Brett Cannon :
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Python-bugs-list
Brett Cannon added the comment:
OK, I'm waiting on issue #14657 (avoiding the _frozen_importlib/importlib
dichotomy) is resolved before I document the new imp.extension_suffixes() as it
would be good to know where I should pull in the source and bytecode suffixes.
I think this only l
Brett Cannon added the comment:
FYI, I just deleted PC/import_nt.c as it stopped compiling after my removal of
_imp.get_suffixes(). Obviously hg has the history of the file so if someone
needs to resurrect it there shouldn't be much
Brett Cannon added the comment:
+1
--
___
Python tracker
<http://bugs.python.org/issue14724>
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.pyth
Brett Cannon added the comment:
The real problem becomes the issue of what zipfile depends on, which
complicates bootstrapping. I mean things could go as far as to write a script
that takes in anchor points in the stdlib and freezes all code they depend on,
but that seems like potential
Brett Cannon added the comment:
To respond to Nick's "yes, there are two copies of importlib._bootstrap"
leanings, distutils2 has actually run into issues with this because they
initially made some assumptions about consistency in what importlib returned
vs. what import doe
Brett Cannon added the comment:
Only two comments, otherwise LGTM (and I can't believe the solution was to go
back through the import system just to pull out the cached module; the things
we would change if we were doing this from scratch).
One, you have some "XXX False" marke
Brett Cannon added the comment:
I think it's beyond a hint and says we need to find a solution or else other
people will run into similar issues.
And while I'm thinking about it, there is precedent for exposing modules under
a different name than they are actually installed as in
Brett Cannon added the comment:
So I was going to try to figure out the logic, so I manually created the test
files to start debugging, but I didn't get the ImportError but instead the 1/0
error for the relative import. Maybe it's specific to lack of threads or the
change you made?
Brett Cannon added the comment:
Should we have a separate context manager for this, or just make it a flag for
a unified import_state() decorator? Or do we want to *always* force the use of
the Python code instead of the frozen code?
--
___
Python
Brett Cannon added the comment:
I would say test_importlib and test_imp (test_import really should just get
folded into test_importlib).
--
___
Python tracker
<http://bugs.python.org/issue14
Changes by Brett Cannon :
--
assignee: -> brett.cannon
resolution: -> fixed
stage: -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python
Brett Cannon added the comment:
Yes.
And the [:] copies the list so I don't accidentally mutate in-place (did that
once already, so now I'm just being paranoid; I doubt I need it hardly anywhere
I don't do +=).
As for using SOURCE_SUFFIXES[0], I'm done following the p
New submission from Brett Cannon :
If you run test.test_warnings it reports that warnings.filters changed. If you
comment out the import of c_warnings in that module, the warning goes away
(this happens with running on tests).
It looks like _PyWarnings_Init() itself doesn't trig
Brett Cannon added the comment:
I have importlib.find_loader() coded up, but getting rid of find_module() is
going to be a pain thanks to pkgutil, multiprocessing.forking, modulefinder,
and idlelib.EditorWindow.
--
___
Python tracker
<h
Changes by Brett Cannon :
--
stage: -> patch review
___
Python tracker
<http://bugs.python.org/issue14796>
___
___
Python-bugs-list mailing list
Unsubscri
New submission from Brett Cannon :
With importlib.find_loader() now implemented, imp.find_module() (and its
companion, imp.load_module()) can go away and stop inflicting their bad API
upon the world.
The trick, though, is fixing code that uses the API. That means pkgutil
Changes by Brett Cannon :
--
dependencies: +Deprecate imp.find_module()/load_module()
___
Python tracker
<http://bugs.python.org/issue13959>
___
___
Python-bug
Brett Cannon added the comment:
http://hg.python.org/lookup/59870239813c documents imp.find_module/load_module
as deprecated w/o actually raising the deprecation. The code works fine, but
the API is just crap. So in the name of ease of transition (and my own personal
sanity), I didn'
Brett Cannon added the comment:
BTW, I'm leaving this issue open in case someone wants to take a stab and
removing the uses of imp.find_module()/load_module() from the stdlb.
--
___
Python tracker
<http://bugs.python.org/is
Brett Cannon added the comment:
I don't feel the need to, but I can in a few days if you want me to (just let
me know if you do).
--
___
Python tracker
<http://bugs.python.org/i
Brett Cannon added the comment:
So from what I can tell you are advocating not resetting anything by default
but instead only save the details and then reset it later. That's fine with me
(aligns more with the warnings context manager), the importlib tests will
simply need to be updat
Brett Cannon added the comment:
sys.modules is tricky thanks to built-in modules not liking to be re-imported
(and why import_state in importlib.test didn't reset it).
If only one test fails because of my assumption, then I guess blanking them out
really isn't as important as I th
Brett Cannon added the comment:
Realized that any decorator or context manager that is created for swapping
_frozen_importlib/importlib._bootstrap should also verify no module is left in
sys.modules with a bad loader and that sys.path_importer_cache doesn't have a
bad finder (I woul
Brett Cannon added the comment:
I see three options (which can be combined). One is to keep using something
like the uncache context manager to make sure the expected modules get removed.
Two is to check the keys of sys.modules at the end of a context manager and if
there are new keys either
Brett Cannon added the comment:
The docs for 2.7 are a little more clear:
http://docs.python.org/install/index.html#inst-search-path says it is not
recursive. And looking at the code
(http://hg.python.org/cpython/file/086afe7b61f5/Lib/site.py#l184) backs that up.
--
resolution
Brett Cannon added the comment:
You can't have it both ways. If you explicitly import __init__ then it becomes
just another module to Python, but you still need the implicit package module
(i.e. without the __init__ name) for everything else to work since so much of
the import system r
Brett Cannon added the comment:
If you can come up with a patch to fix this in a way that is not messy then I
will be happy to look at it.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
If you directly import __init__ then it would just be a module within the
package (the "magic" of packages should stay with the implicit interpretation
of __init__).
--
___
Python tracker
<http://bu
Brett Cannon added the comment:
I see what you mean about the discrepancy, but you don't need to complicate the
constructor to get the desired result. If you have is_package() check if the
module name ends in __init__ to skip the package check and just say False (e.g.
only if the path
Brett Cannon added the comment:
The information is still there, just in a different output line (i.e.
http://hg.python.org/cpython/file/c7b16e2be71a/Lib/importlib/_bootstrap.py#l735
outputs the same info, just on its own line). I couldn't keep the old format as
the code has been sh
3201 - 3300 of 5934 matches
Mail list logo
