On Nov 17, 2007 8:14 PM, Kent Johnson <[EMAIL PROTECTED]> wrote: > Marc Tompkins wrote: > > My question is this: does anybody know of an equivalent to > > "readlines(sizehint)" for non-delimited, binary files? I've Googled > > and Googled until I'm groggy, but I don't seem to find what I want. > > Have you tried specifying a buffer size in the open() call? > > Kent >
Yes. I compared: - no buffer size specified - any of a wide range of positive numbers (from 1 to 4M) - -1 and saw no noticeable difference - as opposed to adding the StringIO buffering, which kicked things up by a notch of six or so. By the way, this is really obscurely documented. It took me a lot of Googling to find even one mention of it - in Programming Python by Mark Lutz - and I was very excited... until I tested it and found that it did nothing for me. Bummer. Then I re-read the passage: > *Buffer size* > > The open call also takes an optional third buffer size argument, which > lets you control stdio buffering for the file -- the way that data is > queued up before being transferred to boost performance. If passed, means > file operations are unbuffered (data is transferred immediately), 1 means > they are line buffered, any other positive value means use a buffer of > approximately that size, and a negative value means to use the system > default (which you get if no third argument is passed, and generally means > buffering is enabled). The buffer size argument works on most platforms, but > is currently ignored on platforms that don't provide the sevbuf system > call. > I've only tested on Windows XP; is XP one of those that don't provide sevbuf? (Actually, I think that's a typo - I think it should be "setvbuf" - but it exists in both the 2001 and 2006 editions of the book.) Perhaps someone who codes closer to the silicon can enlighten me on that score. I just realized that the one thing I didn't try was passing a value of 0 to turn buffering OFF - if, as the above passage seems to suggest, it's always on by default. I might try that tomorrow. But in any case it looks like all it'll do is make things slower. -- www.fsrtechnologies.com
_______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
