On 2004-12-09, Brad Tilley <[EMAIL PROTECTED]> wrote:
> Steven Bethard wrote:
>>>> for root, files, dirs in os.walk(path)
>>>> for f in files:
>>>> try:
>>>> x = file(f, 'rb')
>>>> data = x.read()
>>>> x.close()
>>
>>
>> Remember that CPython is implemented in C, and so all the builtin types
>> (including file) basically execute C code directly. My experience with
>> Python file objects is that they are quite fast when you're doing simple
>> things like the example above.
>
> I'm dealing with a terabyte of files. Perhaps I should have mentioned that.
And you think you're going to read the entire file consisting
of terabytes of data into memory using either C or Python?
[That's the example you gave.]
Sounds like maybe you need to mmap() the files?
Or at least tell us what you're trying to do so we can make
more intelligent suggestions.
--
Grant Edwards grante Yow! I invented skydiving
at in 1989!
visi.com
--
http://mail.python.org/mailman/listinfo/python-list
