[Python-Dev] import too slow on NFS based systems

2012-06-21 Thread Daniel Braniss
Hi,
when lib/python/site-packages/ is accessed via NFS, open/stat/access is very
expensive/slow. 

A simple solution is to use an in memory directory search/hash, so I was
wondering if this has been concidered in the past, if not, and I come
with a working solution for Unix (at least Linux/Freebsd) will it be 
concidered.

thanks,
danny


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] import too slow on NFS based systems

2012-06-21 Thread Daniel Braniss
> On Thu, 21 Jun 2012 13:17:01 +0300
> Daniel Braniss  wrote:
> > Hi,
> > when lib/python/site-packages/ is accessed via NFS, open/stat/access is very
> > expensive/slow. 
> > 
> > A simple solution is to use an in memory directory search/hash, so I was
> > wondering if this has been concidered in the past, if not, and I come
> > with a working solution for Unix (at least Linux/Freebsd) will it be 
> > concidered.
> 
> There is such a thing in Python 3.3, although some stat() calls are
> still necessary to know whether the directory caches are fresh.
> Can you give it a try and provide some feedback?

WOW!
with a sample python program:

in 2.7 there are:
stats   open
27369037
in 3.3
288 57

now I have to fix my 2.7 to work with 3.3 :-)

any chance that this can be backported to 2.7?

cheers,
danny


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com