Branko Badrljica <bran...@avtomatika.com> posted 4a12af80.8040...@avtomatika.com, excerpted below, on Tue, 19 May 2009 15:09:20 +0200:
> Which is more or less ext4 performance if files remain in RAM. When > memory pressure increases, ext4 wil just start writing to the files, > which can actually be faster than tmpfs <--> swap transfers... You bring up a very interesting point. I wonder if anyone has benchmarked both in-memory and to-disk operation of ext4 with extreme write-out times and data=writeback, as compared to tmpfs, hitting swap for disk. Now let me be plain, I wouldn't trust valuable data I need to keep around to ext4 yet, unless I was certain of my backups. (Of course if it's valuable data you want to keep around, you have and have tested your backups in any case, AND keep them updated to the point at which you'd be comfortable losing data, whether that be a few minutes/hours worth or six months worth, but there's a LOT of folks that simply do not have backups at all, and ext4 isn't yet something I'd recommend to them unless they don't care about losing the data at all.) Maybe by 2.6.31, if .30 ext4 is demonstrated stable enough not to be still taking big tweaks in .31, but that's not yet the case with .29 as .30 has some pretty big changes. However, that's NOT what we're talking here AT ALL. Here, we're talking specifically temporary data, intended to be wiped before reboot, such that we don't care if it ever hits disk at all. For that, ext4 is definitely MORE than acceptable at this point. Here's the deal, tho. With tmpfs, since the devs know it's not intended to hit disk /at/ /all/ unless as swap, there's whole sections of normal filesystem code that simply don't apply at all. That's not a /lot/ of overhead when we're talking writing to the slow disk anyway, but it's a bit more significant when we're talking memory-speed operations. That, and the flexibility of tmpfs to use swap when necessary, has traditionally made it an extremely good choice for such temporary data. If we stop right there, ext4 would still have higher overhead, even with delayed write, and would thus be a characteristically worse choice. HOWEVER, and a BIG HOWEVER it is, there's a a particular caveat with swap where ext4 gets an extremely good mark, that may well turn the tables. The problem with swap in general, and thus anything that ends up using it, is that it has sort of been the ugly stepchild no one has bothered to do a whole lot with. The attitude of the kernel devs has seemed to be that with memory so cheap these days, there should be no reason to be using swap under normal operating conditions anyway; that it's only an extreme worst-case solution to exercise right before the OOM-killer goes to work, and that as a result, any serious work optimizing it beyond sane minimums is "wasted". This has of course been somewhat controversial, particularly for users of older systems with < a gig of memory capacity, total, and only a single disk, but the fact is, swap /has/ been a somewhat ignored subsystem. Obviously, ext4 is anything /but/ ignored. Thus, it might actually be worth the (at least in theory) somewhat higher memory overhead if the disk access is significantly more optimized than swap disk access is, and as explained above, that's a reasonably likely case. Of course, as you stated earlier in passing but no one has actually fleshed out, this assumes anything BUT default ext4 writeout expiry parameters. We're talking expiry times on the order of an hour, instead of the default few seconds, thus keeping data in memory for the entire emerge, so the whole working dir can be created, the sources unpacked to it, the binaries compiled and whatever else needs done to the sources, fake-installed, qmerged to the live filesystem, then the whole thing, scratch files, sources, working dir, all of it, deleted, before the ext4 write-out expiry occurs, assuming no memory pressure to cause the writeouts earlier. But it's a valid question to raise and one I've seen no discussion of, anywhere, certainly not in the LWN and HOnline coverage that's where I get most of my kernel information from. Under that sort of explicit temporary use only tweaking of the ext4 tuning knobs, exactly how /does/ it compare to tmpfs, both under memory-only scenarios and under memory- pressure-therefore-swapped-or-written-out scenarios. Benchmarks are definitely called for, as it's a very legitimate and unsettled question, going on theory alone. Of course, in a typical single-disk usage scenario, with both a swap and this dedicated ext4 temp partition we're talking about, it would very likely depend on the partition ordering, with whichever partition is closest to the outside of the disk getting the better marks as the physical disk I/O will be faster to it. But, for the benchmark the two partitions could be swapped, and that compared against having the OTHER one on a separate device, say swap on a different device when testing the ext4 partition, just so it didn't interfere. Another typical case-by-case factor would be what other data is read/ written and how close it is to the partition in question. (For our case, the Gentoo tree and sources and system header, etc files, read, the package database, live filesystem destinations, and binpkg location if that feature is turned on, write, and ccache both r/w if it's on, are the system data that would be accessed during the build.) That sort of thing would have to be kept in mind and controlled for as well, in addition to the usual effort at keeping unrelated process activity to a minimum during the benchmark to avoid screwing the outcome. Sounds interesting to think and read about, if someone has a link. But honestly, I don't personally care enough about it to do the benchmarks or even to spend time looking to see if there are any, myself. So, does it sound like fun to anyone? For all I know someone's already tested it. I haven't been following ext4 development close enough to know for sure, if I didn't simply happen across it. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
