Brett Cannon added the comment:
On Mon, Feb 6, 2012 at 17:06, STINNER Victor wrote:
>
> STINNER Victor added the comment:
>
> Is there a benchmark for import? How slow is importlib? :)
>
importlib.test.benchmark
--
___
Python
Brett Cannon added the comment:
Brian, what is left for updating your patch? I'm going to need this to fix
test_pydoc to not fail in my importlib bootstrap work so I can (finally) help
with this. Is it just actually using the new features of the exception?
--
assignee: brett.c
Brett Cannon added the comment:
On Thu, Feb 9, 2012 at 00:03, Brian Curtin wrote:
>
> Brian Curtin added the comment:
>
> Yep, I just need to actually make use of the feature. I'll generate a new
> patch shortly.
>
If you can generate it before 17:30 EST today I ca
Brett Cannon added the comment:
Thanks, Brian! I'll do a review tonight on the drive home (and maybe even write
the docs up).
--
___
Python tracker
<http://bugs.python.org/issu
Brett Cannon added the comment:
With issue1559549 adding a 'name' argument, I'm going to push to have it gain a
reasonable default __str__ if 'name' is set but nothing else. That patch also
contains some helper functions which should simplify Eric's p
Brett Cannon added the comment:
The patch looks fine for its current semantics except for the fact that the
macro doesn't work under clang; ##macro_var is supposed to be used only when
concatenating parts of a string to create a complete identifier, not
concatenating two identifiers ne
Brett Cannon added the comment:
And to answer David's joke, I carpool from Toronto to Waterloo four days a week
so I have an hour each direction to work on stuff.
--
___
Python tracker
<http://bugs.python.org/issu
Brett Cannon added the comment:
Oh, and there are no forward declarations for the new functions added to
errors.c
--
___
Python tracker
<http://bugs.python.org/issue1559
Brett Cannon added the comment:
Attached is Brian's patch with the macro fix and forward declarations.
--
Added file: http://bugs.python.org/file24476/importerror.diff
___
Python tracker
<http://bugs.python.org/issu
Brett Cannon added the comment:
So I just tried to pass fullname in import.c and it didn't work (of course).
Looks like even when I try to use fullname in find_module_path_list() and
load_next() it is still leaving out the package name. Ugh. That means either
refactoring import.c to ge
Brett Cannon added the comment:
It looks like 2to3 isn't being converted properly. Benjamin, is lib2to3
supposed to work on itself and result in a sane output? If not then the
benchmark should probably drop its internal copy of lib2to3.
Antoine (since you last looked at the benchmark
Brett Cannon added the comment:
Just a quick update. I have refactored importlib in the cpython repo to allow
for implementing bits of importlib.__import__() and importlib._gcd_import() in
C. This means that the built-in __import__() is now calling importlib
underneath the covers. Eventually
Brett Cannon added the comment:
Add Nick since the refactoring of importlib.__import__() into functions was his
idea.
--
nosy: +ncoghlan
___
Python tracker
<http://bugs.python.org/issue2
Brett Cannon added the comment:
Patch looks fine, although there aren't any tests. At least for importlib you
can simply use a finder test for extension modules to verify the warning is
triggered. That way you can create an empty module with the expected naming
rather than worry
Brett Cannon added the comment:
So I simply swapped out the code and the tests fail. Then I realized why: while
the assumption is right, that does not mean that that name passed to
__import__() isn't relative and thus shifts what need to be returned (the else
clause case). That's
Brett Cannon added the comment:
Thanks for the patch, Charles-François!
--
resolution: -> fixed
stage: commit review -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Brett Cannon added the comment:
It's not working because when you import a.b you calculate __path__ at import
time, so when you modify sys.path it won't make a difference since import will
look at a.__path__ after your a.b import and simply ignore sys.path. But when
you put eve
Brett Cannon added the comment:
I have not had a chance to deep-dive in the patch, but I just wanted to
double-check that this (a) implemented the idea PJE and I discussed on
python-dev, and (b) you re-generate the patch as I pushed some changes to
importlib today
Brett Cannon added the comment:
Just to fill you in, the discussion centred on the idea of doing a listdir() of
the directory the FileFinder was in charge of watching and caching that. Then,
when it had to look up a file all it had to do was stat the directory to look
for a change before it
Brett Cannon added the comment:
OK, I have gone ahead and done a review over on rietveld for the code as-is.
But I do have a design question.
Why pre-calculate everything? In the most common case any single module will be
imported once, if at all. And once it is imported it will get cached
Brett Cannon added the comment:
On Fri, Feb 17, 2012 at 14:31, Antoine Pitrou wrote:
>
> Antoine Pitrou added the comment:
>
> > Why pre-calculate everything? In the most common case any single
> > module will be imported once, if at all. And once it is imported it
&
Brett Cannon added the comment:
And while I'm thinking about it, the hybrid caching approach of caching just
the directory contents to start and then caching full details on successful
searches (and failures) would also let you cache the full file path as the
strings will be interne
Brett Cannon added the comment:
The lastest patch (listdir_cache2) LGTM. Have you by any chance patched it in
and run ./python -m importlib.test.regrtest (runs the entire test suite with
builtins.__import__() substituted with importlib.__import__())? If not I can do
it on my ride home
Brett Cannon added the comment:
Well, it's the reason you added file size to .pyc files. =) I assume if you
store the system clock as well to see if no time has changed it would kill the
performance gain. We could also have two versions of _FileFinder such that
people could choose which
Brett Cannon added the comment:
I wouldn't worry too much about test_runpy as it has always been a touchy test
for me.
--
___
Python tracker
<http://bugs.python.org/is
Brett Cannon added the comment:
I like Martin's suggestion of simply throwing an error. But then again I also
really like his idea of simply not warning since it's easier. =)
--
___
Python tracker
<http://bugs.python.o
Brett Cannon added the comment:
It's surprisingly difficult to get right because tests will fail if you keep
the path relative in other situations (such as test_cmd_line_script). It was an
absolute pain to get it to where it is now. If you can make it keep the
relative path and pass the
Changes by Brett Cannon :
--
resolution: -> fixed
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue14054>
___
___
Python-bugs-list
Changes by Brett Cannon :
--
stage: needs patch -> committed/rejected
___
Python tracker
<http://bugs.python.org/issue14054>
___
___
Python-bugs-list mai
Brett Cannon added the comment:
Since we never solved this I'm still getting compiler warnings. Can we decide
on a solution for this so I can go back to be warning-free (sans
`-Wno-unused-value -Wno-empty-body -Qunused-arguments`)?
/Users/bcannon/Developer/repo/cpython/bootstrap_impo
Brett Cannon added the comment:
Antoine, do you think this may have been triggered by your latest patch?
--
nosy: +pitrou
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
The test suite does use importlib for some imports as finders from importlib
for some packages stick around. I brought this up a way back but people thought
it was actually a good idea to let importlib handle imports to stress test it.
Anyway, if it can'
Changes by Brett Cannon :
--
resolution: -> invalid
status: open -> closed
___
Python tracker
<http://bugs.python.org/issue14052>
___
___
Python-bugs-list
Brett Cannon added the comment:
I have a Mac, but I'm heading out the door; I will see if I can diagnose it and
fix it tonight or tomorrow (unless someone beats me to it).
--
___
Python tracker
<http://bugs.python.org/is
Brett Cannon added the comment:
The patch uses the time module which is not a built-in, so it breaks
bootstrapping by directly importing a module, and an extension at that. At best
you could switch to a modified _FileFinder after importlib is initially running
and able to import extension
Brett Cannon added the comment:
On Mon, Feb 20, 2012 at 18:06, Antoine Pitrou wrote:
>
> Antoine Pitrou added the comment:
>
> > The patch uses the time module which is not a built-in, so it breaks
> > bootstrapping by directly importing a module, and an extension at
Changes by Brett Cannon :
--
nosy: -brett.cannon
___
Python tracker
<http://bugs.python.org/issue8942>
___
___
Python-bugs-list mailing list
Unsubscribe:
Brett Cannon added the comment:
I don't quite follow what you want. You say you want to *ignore* changes to
source files, but that's the whole point of a reload. And I don't see how
ignoring bytecode changes anything unless you are mucking with bytecode formats
and don&#x
Brett Cannon added the comment:
Then I'm not understanding the problem as changes to source trump .pyc files in
a reload(). It should make no difference that .pyc files even exist to a
reload() as long as source is around.
--
status: open ->
Brett Cannon added the comment:
LGTM
--
stage: -> commit review
___
Python tracker
<http://bugs.python.org/issue14077>
___
___
Python-bugs-list mai
Brett Cannon added the comment:
So refactoring the Python code into C code has been done, but it's crashing. =)
As usual, manual memory management sucks.
I also think that their is still too much C code as it makes the whole thing
somewhat brittle to any refactoring of importlib.
Brett Cannon added the comment:
2012/2/21 Charles-François Natali
But in other VMs id() is simply a number that gets assigned to objects that
is monotonically increasing. So it can be extremely deterministic if the
object creation order is consistent
Brett Cannon added the comment:
I think the test is bogus. It would do better to either use str.endswith() or
splitting off the various parts of __file__ to verify the filename is correct
as is the directory. Otherwise it shouldn't matter if the directory is relative
or abs
Brett Cannon added the comment:
Well, if you execute test_imp w/ importlib (-m importlib.test.regrtest
test_imp) it still passes. So there is something very odd going on that
probably relies on some other test doing something weird. And this might be
hard to diagnose until the error triggers
Brett Cannon added the comment:
I can confirm the failure when an import fails prior to running test_imp (i.e.
trying importing some non-existent module like Stefan did at the top of
test_imp). Luckily importlib doesn't fail in this
Brett Cannon added the comment:
I'd rather get importlib working fast enough that you are okay with
bootstrapping it so we can delete the offending code. =)
--
___
Python tracker
<http://bugs.python.org/is
Brett Cannon added the comment:
On Wed, Feb 22, 2012 at 19:35, Éric Araujo wrote:
I have already done that and pushed the initial change. The ultimate goal
will be fetching from sys.modules when level == 0 and not bool(fromlist)
only touch C code. All other work will touch varying levels of
Brett Cannon added the comment:
That last comment from me looks odd because somewhere along the way my replies
to Éric's comment got stripped out.
First two paragraphs are in response to 1)
Third paragraph is for 2)
Fourth paragraph is for 3)
Rest makes sense since the inline reply sur
Brett Cannon added the comment:
OK, so I have now streamlined the case so that only C is called when a module
is in sys.modules, level == 0 and not fromlist.
Using the normal_startup benchmark, bootstrapped importlib is 1.12x slower. A
fast run of startup_nosite puts it at 1.2x slower
Brett Cannon added the comment:
So I finished rewriting _return_module() so that if fromlist is not defined
then it's all in C (and subsequently naming the function _handle_fromlist()).
That leaves rewriting in C _calc___package__() and _resolve_name(). After that
I don't thin
Brett Cannon added the comment:
OK, I have now done as much C code as I'm going to do for the __import__()
function. It has gotten bootstrapped importlib within 10% of normal_startup
against default.
That leaves (possibly) rewriting BuiltinImporter in C and then just good old
fash
Brett Cannon added the comment:
Fixed the 'for' loop variable declaration. Surprised clang didn't warn me about
that.
As for reviews, I'm totally happy to get them, but I also don't know if I have
hit the performance point well enough for people to allow me to merge
Changes by Brett Cannon :
Added file: http://bugs.python.org/file24632/c011ff345a78.diff
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Python-bugs-list m
Brett Cannon added the comment:
Then if you want to give it a review that would be great!
I still need to solve the test_pydoc failure (either with Brian's patch to add
a name attribute to ImportError or implement importlib.find_module()).
Otherwise all other failures at the momen
Brett Cannon added the comment:
Importlib actually requires no files from disk; everything that is required for
importlib are built-in modules or are constants in importlib itself (e.g.
os.sep). So technically this should be doable since my bootstrap work freezes
importlib itself
Brett Cannon added the comment:
importlib.invalidate_caches() would become a permanent thing if the directory
listdir cache approach stays and importlib gets bootstrapped. But it would be
made more general to work for any and all loaders and not be specific to
importlib (e.g. 3rd-party
Brett Cannon added the comment:
Does test.support track the locale for test mutations? If not we might want to
add that check w/ all of the other interpreter state checks we have.
--
nosy: +brett.cannon
___
Python tracker
<http://bugs.python.
Changes by Brett Cannon :
--
nosy: +brett.cannon
___
Python tracker
<http://bugs.python.org/issue14126>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Brett Cannon :
Issue #14113 pointed out that test.support doesn't check if tests mucked with
the locale after running.
--
components: Tests
keywords: easy
messages: 154414
nosy: brett.cannon
priority: normal
severity: normal
status: open
title: check for l
Brett Cannon added the comment:
Created issue14135 for having test.support check the locale to see if it has
been changed.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
Not quite, Michele. So this bug is about taking the stuff that is printed to
stdout/stderr explicitly in parsetok.c and instead using the warnings
functions, e.g. PyErr_Warn(), since _warnings is a built-in module (now) and
thus is available to the tokenizer
Brett Cannon added the comment:
This is issue #14052 all over again, just with a different test.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
Well, we already have a delay in detecting new files in importlib as the
directory mtime check already adds a delay of the granularity of the mtime
value for a directory. So if you manage to mutate a directory, import, write a
file, and then import again all
Brett Cannon added the comment:
Then how do you explain the failure being nothing more than prepending .
compared to an absolute file path?
Anyway, the test passes under importlib with ``./python.exe -m
importlib.test.regrtest test_imp``, so it requires some special setup to
trigger the
New submission from Brett Cannon :
The _io module imports the os module purely for the use of
os.device_encoding(). That function, though, is defined by posix and thus is
already available to C code. To avoid this import dependency it would be better
to simply expose the function somehow so
Brett Cannon added the comment:
Either that or PyObject *_Py_device_encoding(PyObject *fd) would help since
both _io and posixmodule.c will have a PyObject already for the fd.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
OK, then I will make this happen. People care whether it take an int or a
PyObject? That seems to be the only open question.
--
___
Python tracker
<http://bugs.python.org/issue14
Brett Cannon added the comment:
Attached is a patch which is trying to do the refactor, but I'm somehow causing
a something to be gc'ed when it shouldn't. If anyone spots something obvious
let me know, else I will try to debug some more on my way home.
--
keywords: +p
Brett Cannon added the comment:
Victor's point fixed my issue.
As for a test, I will happily write one, but what can I test? A string for fd 0
and None for fd -1 (or is there some os call I can make to find the largest fd
so I can test
Brett Cannon added the comment:
OK, so if on windows or isatty + nl_langinfo + CODESET, check fd 0 for a
returned string that codecs.lookup and find something for. fd 42 (if not a tty)
should return None.
--
___
Python tracker
<h
Changes by Brett Cannon :
--
assignee: -> brett.cannon
resolution: -> fixed
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python
Brett Cannon added the comment:
I like that argument. =) If this is not an issue in Python 3.3 then this should
be closed as out of date since it will break code if it is changed.
--
___
Python tracker
<http://bugs.python.org/issue1346
Brett Cannon added the comment:
I don't care about compatibility between zipimport and importlib.
--
___
Python tracker
<http://bugs.python.org/issu
Brett Cannon added the comment:
I have replied to Antoine's review and so generated a new patch.
At this point my bootstrap_importlib branch is 5% slower in a standard build in
the normal_startup benchmark (11% if you use a debug build).
This is still w/o profiling the Python code to
Changes by Brett Cannon :
Added file: http://bugs.python.org/file24700/131d1d107f26.diff
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Python-bugs-list m
Changes by Brett Cannon :
Removed file: http://bugs.python.org/file24418/f0b459af26fb.diff
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Python-bugs-list m
Brett Cannon added the comment:
The finding/loading code in import.c is purely because of the imp module's
public API (e.g. imp.find_module()). I have been waiting to find out if
importlib would get bootstrapped before making the current module _imp and then
creating an imp.py which ha
Brett Cannon added the comment:
It technically doesn't depend, but it potentially would make this moot. But
stuff is going on at the language summit which is going to shift stuff around.
--
dependencies: -ImportError needs attributes for module and file
Brett Cannon added the comment:
It's quite possible you are right, Michele. I don't know if anyone has looked
at what exactly is required for _warnings.c compared to pgen.
--
___
Python tracker
<http://bugs.python.org
Brett Cannon added the comment:
Two things I realized yesterday that need to be implemented (and some
double-checking and/or opinion would be nice) while I wait for a full patch
review.
One is ``python -v`` support. sys.flags has a verbose attribute that can be
used to properly guard
Brett Cannon added the comment:
So after looking at import.c and how it handles stuff, I have decided how I
already implemented __import__() triggering __import__() itself is fine. So
Antoine can just stay confused by my comment as it is now a moot point
Brett Cannon added the comment:
Looks like Windows as a PC/import_nt.c:_PyWin_FindRegisteredModule() function
used to find modules in the registry. Probably should care about that at some
point (I guess).
--
___
Python tracker
<h
Changes by Brett Cannon :
--
nosy: +eric.snow
___
Python tracker
<http://bugs.python.org/issue13959>
___
___
Python-bugs-list mailing list
Unsubscribe:
Brett Cannon added the comment:
Attached is a start for re-implementing imp. Still need code for
cache_from_source, source_from_cache, find_module, load_module, reload,
load_compiled, load_source, and load_package.
--
keywords: +patch
Added file: http://bugs.python.org/file24846
Brett Cannon added the comment:
Yes, that's the important bit. Thomas has given it a once over at this point so
you should be able to see his review comments as well to prevent duplicate work.
--
___
Python tracker
<http://bugs.py
Brett Cannon added the comment:
Thanks to everyone for the reviews. I'm moving house tomorrow and I suspect
Andrea wants me to take a little Python break so I might not get to the reviews
before a2, but I will definitely get to them an merged
Brett Cannon added the comment:
Cursory glance has the patch LGTM.
--
___
Python tracker
<http://bugs.python.org/issue14331>
___
___
Python-bugs-list mailin
Changes by Brett Cannon :
Added file: http://bugs.python.org/file25104/0fdf350657e3.diff
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Python-bugs-list m
Brett Cannon added the comment:
OK, everyone's code review comments have been addressed. That means I have
test_pydoc still failing (and that won't get fixed until ImportError grows a
module_name attribute; issue #1559549) and test_trace (it doesn't like frozen
modules).
Changes by Brett Cannon :
--
resolution: -> fixed
stage: needs patch -> committed/rejected
status: open -> closed
___
Python tracker
<http://bugs.python.or
Brett Cannon added the comment:
OK, -v/PYTHONVERBOSE is as done as it is going to be by me. Next up is
(attempting) Windows registry stuff. After that I will push to default with
test_trace and test_pydoc skipped so others can help me with those
Brett Cannon added the comment:
On Fri, Apr 6, 2012 at 16:05, Antoine Pitrou wrote:
>
> Antoine Pitrou added the comment:
>
> > OK, -v/PYTHONVERBOSE is as done as it is going to be by me. Next up is
> > (attempting) Windows registry stuff. After that I will pus
Brett Cannon added the comment:
OK, I have fixed test_trace by tweaking the trace module in default. I have
decided to follow Antoine's suggestion-by-question and get test_pydoc working
before I merge by getting ImportError spruced up in default but pydoc changed
in bootstrap_importlib.
Changes by Brett Cannon :
--
dependencies: +ImportError needs attributes for module and file name
___
Python tracker
<http://bugs.python.org/issue2377>
___
___
Brett Cannon added the comment:
This is now officially blocking my importlib bootstrap work (issue #2377), so I
have assigned this to myself to get done and I hope to get this committed
within a week *without* updating Python/import.c and the requisite tests or
pydoc (which I will make part
Brett Cannon added the comment:
Once importlib bootstrapping lands and I expose the rest of the API you can
replace imp.load_source() w/ importlib.SourceFileLoader(name,
path).load_module(name) and get the same result.
--
___
Python tracker
<h
Changes by Brett Cannon :
--
assignee: brian.curtin -> brett.cannon
___
Python tracker
<http://bugs.python.org/issue1559549>
___
___
Python-bugs-list mai
Brett Cannon added the comment:
Patch for the attributes is in (no use by import.c)! I'm going to do a separate
commit for use by importlib and then fix pydoc in my bootstrap branch.
--
resolution: -> fixed
stage: patch review -> committed/rejected
status: ope
Brett Cannon added the comment:
Just to answer Greg's question, importlib uses a context manager to manage the
import lock so as long as that surfaces the right thing in a fork then things
will be fine.
--
___
Python tracker
New submission from Brett Cannon :
Because I don't have access to Windows, importlib doesn't check the Windows
registry for paths to search (see the use of _PyWin_FindRegisteredModule() in
Python/import.c and its definition in PC/import_nt.c).
I am considering this a release block
2001 - 2100 of 5934 matches
Mail list logo
