On Tue, 2010-10-26 at 10:36 +0200, Sven Joachim wrote: > On 2010-10-21 19:14 +0200, Jonathan Nieder wrote: > > > Ken Bloom wrote: > > > >> And what mount options are you using? If you're using > >> defaults, /etc/mtab (and therefore the mount command) won't know what > >> the default values are, but you can check /proc/mounts which will > >> include the data= mount option. > > > > data=ordered. That's the default for ext4. > > Yes, and data=writeback does not make much of a difference. However, > using the "nodelalloc" mount option does wonders, increasing unpacking > speed in dpkg 1.15.7 by a factor of ~8. > > Sven
So here's the upshot: Delayed allocation is supposed to make your filesystem blazingly fast, but in order to get data safety you *need* to call fsync() to make sure your data's on disk[1]. However, fsync() causes real performance hit, made worse by the fact that you can only call fsync() on one file at a time, so to fsync() a whole unpacked .deb, you have to block on fsync() possibly hundreds of times. And it seems that fsync() is only writing one file to disk at a time, as it should be. If you turn off delayed allocation, then filesystem operations slow down in general, but fsync() gets faster somehow (I'm not sure why), and you get greater data safety in the first place. I think the proper thing to do at this point is come up with a concise summary of the different options you've tried, and their performance, and send that to the Linux Kernel Mailing List (linux-ker...@vger.kernel.org)[2] and ask them what your options are, what their performance characteristics are, and what their safety characteristics are. And also the tradeoffs of using various mount options. Ask about ext3, ext4, and btrfs. We can decide later whether ext3 really concerns us. (If you email LKML, then please don't CC: any of us, and don't CC: the bug report. LKML threads can be very high traffic. You can look up the message you sent in the LKML archives at lkml.org, and send the URL of your message to the BTS.) --Ken [1] However, since 2.6.30, ext4 will make cause a file to have any delayed allocation blocks to be allocated immediately when a file is replaced, which I think is dpkg's use case. I think that means you should be able to get the safety you seek without calling fsync() at all. See http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ [2] I don't expect debian-kernel has the expertise to answer these questions. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
