On Tue, May 27, 2014 at 11:12 AM, J. Roeleveld <jo...@antarean.org> wrote: > On Tuesday, May 27, 2014 10:31:26 AM Rich Freeman wrote: >> btrfs wouldn't have any issues with this at all. You'd have an >> advantage in that you wouldn't have to unmount the filesystem to >> cleanly create the snapshot (which you have to do with lvm). > > That, or a "sync" prior to creating the snapshot. :)
If the filesystem is still mounted, I'm not sure that a sync is guaranteed to give you a clean remount. It only flushes the caches/etc. You need to remount read-only or unmount before doing the sync (and the sync probably isn't actually necessary as I'd think LVM would snapshot the contents of the cache as well). > I have a yearly (full), monthly, weekly and daily. Each incremental is against > the most recent one of itself or longer period. > That means having to keep multiple snapshots active, which I prefer to avoid. You only need to store snapshots for use with incremental backups. So, if all your backups are full, then you don't need to retain any snapshots (and you wouldn't use btrfs send anyway). If your yearly is full and your monthlies are incremental against the yearly then you need to keep your yearly snapshot for a year. If your yearly is full and your monthlies are incremental against the last month, then you only need to keep the yearly until the next monthly. If your monthlies are full then you only need to keep the current monthly assuming your dailies are incremental against those, but if they're incremental from the last daily then you never need to keep anything for more than a day. > > But, it is a good idea for backing up desktops and laptops. It is really intended more for something like datacenter replication. Snapshot every 5 min, send the data to the backup datacenter, delete the snapshots upon confirmation of successful receipt. In such a scenario you wouldn't retain the sent files but just keep playing them against the replicate filesystem. They'd be fine for backups as well, as long as you can store the snapshots online until no longer needed for incrementals. > >> But, you can always just create a snapshot, write it to backup with >> your favorite tool (it is just a directory tree), and then remove it >> as soon as you're done with it. Creating a snapshot is atomic at the >> filesystem level, though again if you want application level >> consistency you need to deal with that until somebody comes up with a >> transactional way to store files on Linux that is more elegant that >> fsyncing on every write. > > That would require a method to keep database and filesystem perfectly in sync > when they are not necessarily on the same machine. > Well, right now we can't even guarantee consistency when everything is written by a single process on the same machine. The best we have is a clunky fsync operation which kills the write cache and destroys performance, and even that doesn't do anything if you have more than one file that must be consistent. The result is journals on top of journals as nobody can trust the next layer down to do its job correctly. Going across machines does complicate things further as there are more failure modes that take out one part of the overall system but not another. However, I'd like to think that an OS that natively supports transactions could at least standardize things so that every layer along the path isn't storing its own journal. In fact, many of the optimizations possible with zfs and btrfs are due to the fact that you eliminate all those layers. Rich
