On Mon, Apr 29, 2013 at 1:46 AM, Jerry DeLisle <jvdeli...@charter.net> wrote: > OK Janne and thanks for the patch.
Thanks for the review, committed (as well as the system_clock patch). > What are your thoughts about special casing nul devices/ Hmm, I'm not that eager. It starts to smell of "benchmarketing".. One thing I have been thinking of which could help would be to implement a "start of the current record" marker in the buffering implementation, and when flushing then only flush up to that marker. Currently when writing small sequential unformatted records what often happens when looking at strace output is something like write(3, "\4\0\0\0`\0263I\4\0\0\0\4\0\0\0p\0263I\4\0\0\0\4\0\0\0\200\0263I"..., 8192) = 8192 write(3, "\4\0\0\0", 4) = 4 lseek(3, 8810688, SEEK_SET) = 8810688 write(3, "\4\0\0\0", 4) = 4 lseek(3, 8810700, SEEK_SET) = 8810700 write(3, "\4\0\0\0\20A3I\4\0\0\0\4\0\0\0 A3I\4\0\0\0\4\0\0\0000A3I"..., 8192) = 8192 i.e. the buffer fills up in the middle of a record, we flush it (the 8192 byte writes), but then we have to seek back and forth to fix the record markers. This also means that we cannot fully buffer non-seekable files (such as /dev/null) because the records around the buffer boundaries are corrupted (which of course doesn't matter for the particular case of /dev/null, but otherwise..). So if this issue is fixed we could buffer those as well and get essentially the same performance as for regular files. E.g. something like /* Reserve space in the buffer (flush existing data if necessary), up to some reasonable max size (e.g. 4 KB) unless the size is small and known upfront (e.g. direct access), and set the current_record_start marker at the current position. Should be called when preparing a new record (st_write()). */ void sreserve(int size); Write the record data as usual... /* Finish the record, i.e. set current_record_start marker to -1 to mark that there is no current record. size should be <= reserved size. Should be called when finishing a write (st_write_done()). */ void scommit(int size); Well, that's a rough idea. I don't know when or if I'll have time and motivation to implement it, though.. -- Janne Blomqvist
