Thouis (Ray) Jones added the comment:
Further testing indicates the problem is in the filesystem itself (either the
server or client, but not in python).
Serializing the loops calling readdir / readdir_r fixes the problem on my
system, but using either function in a large number of parallel
Thouis (Ray) Jones added the comment:
> > It's also possible that readdir() is not reentrant with lstat()
> This doesn't make much sense to me.
Me either. I think what I was actually seeing was multiple calls to readdir()
still occurring even after placing a mutex on o
Thouis (Ray) Jones added the comment:
Reading through many pages discussing readdir vs. readdir_r (many on security
mailing lists, a large number referring to the page linked in the patch), I get
the impression that most implementations are thread-safe as long as separate
threads do not call
Changes by Thouis (Ray) Jones :
Added file: http://bugs.python.org/file23834/py272_readdir_r.v2.patch
___
Python tracker
<http://bugs.python.org/issue13517>
___
___
Pytho
Thouis (Ray) Jones added the comment:
I should add the caveat that I am not completely confident that I have
stress-tested the patch enough to be sure that it actually addresses the
problem. It is still possible that this is an error in OSX or the remote
fileserver in which a large amount
Thouis (Ray) Jones added the comment:
Here is the script I use to detect the failure.
% python filefinder.py /PATH/TO/LARGE/DIRECTORY/TREE
(note that I was working over samba with an 8ish-level deep directory with
around 25 files).
Compare its final output in the FOUND column with
New submission from Thouis (Ray) Jones :
On my system (OSX 10.6.8) using the python.org 32/64-bit build of 2.7.2, I see
incorrect results from os.listdir() in a threaded program. The error is that
the result of os.listdir() is missing a few files from its list.
First, my use case. I work
Changes by Thouis (Ray) Jones :
--
nosy: +thouis
___
Python tracker
<http://bugs.python.org/issue7946>
___
___
Python-bugs-list mailing list
Unsubscribe:
