Hello
Speaking from experience from my observations on millions of machines the
stat() call is *very slow* when compared to readdir(), FindNextFile(),
getdirentriesattr(), etc. When we switched from a file system indexer that
stat()ed every file to one that read directories we noticed an averag
On 1/31/2011 1:38 PM, Brett Cannon wrote:
> I should mention that I have considered implementing a caching finder
> and loader for filesystems in importlib for people to optionally
> install to use for themselves. The real trick, though, is should it
> only cache hits, misses, or both? Regardless,
> Another thing to consider: on App Engine (which despite of all its
> architectural weirdness uses a -- mostly -- standard Linux filesystem
> for the Python code of the app) someone measured that importing from a
> zipfile is much faster than importing from the filesystem. I would
> imagine this e
On Mon, Jan 31, 2011 at 04:43, Antoine Pitrou wrote:
> On Mon, 31 Jan 2011 00:08:25 -0800
> Guido van Rossum wrote:
>>
>> (Basically I am biased to believe that stat() is a pretty slow system
>> call -- this may just be old NFS lore though.)
>
> I don't know about NFS, but starting a Python inter
On Jan 30, 2011, at 05:35 PM, Victor Stinner wrote:
>And the real question is: should we change that before 3.2 final? If we
>don't change that in 3.2, it will be harder to change it later (but it
>is still possible).
I don't see how you possibly can without re-entering beta. Mucking with the
im
On Mon, 31 Jan 2011 00:08:25 -0800
Guido van Rossum wrote:
>
> (Basically I am biased to believe that stat() is a pretty slow system
> call -- this may just be old NFS lore though.)
I don't know about NFS, but starting a Python interpreter located on a
Samba share from a Windows VM is quite slow
On 2011-01-30 21:43, "Martin v. Löwis" wrote:
Am 30.01.2011 17:54, schrieb Alexander Belopolsky:
On Sun, Jan 30, 2011 at 11:35 AM, Victor Stinner
wrote:
..
We should find a compromise between speed (limit the number of system
calls) and the usability of Python modules.
Do you have measureme
On Sun, Jan 30, 2011 at 11:33 PM, "Martin v. Löwis" wrote:
>> Maybe also
>>
>> * Read and cache the directory contents and search it ourselves
>> instead of making a system call for every possible name.
>
> I wouldn't do that - I would expect that this is actually slower than
> making the
> Maybe also
>
>* Read and cache the directory contents and search it ourselves
> instead of making a system call for every possible name.
I wouldn't do that - I would expect that this is actually slower than
making the system calls, because the system might get away with not
reading the
Victor Stinner wrote:
Limit the number of suffixes is maybe not the right solution to limit
the number of system calls at startup. We can imagine alternatives:
* remember the last filename when loading a module and retry this
filename first
* specify that a module is a Python system module an
Am 30.01.2011 17:54, schrieb Alexander Belopolsky:
> On Sun, Jan 30, 2011 at 11:35 AM, Victor Stinner
> wrote:
> ..
>> We should find a compromise between speed (limit the number of system
>> calls) and the usability of Python modules.
>
> Do you have measurements that show python spending signif
On Sun, Jan 30, 2011 at 11:35 AM, Victor Stinner
wrote:
..
> We should find a compromise between speed (limit the number of system
> calls) and the usability of Python modules.
Do you have measurements that show python spending significant time on
failing open calls?
_
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Am 30.01.2011 17:35, schrieb Victor Stinner:
> Le dimanche 30 janvier 2011 à 22:52 +1000, Nick Coghlan a écrit :
>> On Sun, Jan 30, 2011 at 7:25 PM, Georg Brandl wrote:
And why it does test with and without "module".
>>>
>>> Because it always did
Le dimanche 30 janvier 2011 à 22:52 +1000, Nick Coghlan a écrit :
> On Sun, Jan 30, 2011 at 7:25 PM, Georg Brandl wrote:
> >> And why it does test with and without "module".
> >
> > Because it always did (there's a thing called backwards compatibility.)
> >
> > This is of course probably the obvio
On Sun, Jan 30, 2011 at 7:25 PM, Georg Brandl wrote:
>> And why it does test with and without "module".
>
> Because it always did (there's a thing called backwards compatibility.)
>
> This is of course probably the obvious one to start a deprecation process.
But why do we check the long suffix fo
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Am 30.01.2011 09:56, schrieb Victor Stinner:
> Hi,
>
> Antoine Pitrou noticed that Python 3.2 tries a lot of filenames to load
> a module:
> http://bugs.python.org/issue11051
>
> Python 3.1 does already test many filenames, but with Python 3.2, it is
> Python 3.1 does already test many filenames, but with Python 3.2, it is
> even worse.
>
> For each directory in sys.path, it tries 9 suffixes: '',
> '.cpython-32m.so', 'module.cpython-32m.so', '.abi3.so',
> 'module.abi3.so', '.so', 'module.so', '.py', '.pyc'.
>
> I don't understand why it tests
Hi,
Antoine Pitrou noticed that Python 3.2 tries a lot of filenames to load
a module:
http://bugs.python.org/issue11051
Python 3.1 does already test many filenames, but with Python 3.2, it is
even worse.
For each directory in sys.path, it tries 9 suffixes: '',
'.cpython-32m.so', 'module.cpython-
18 matches
Mail list logo
