On 6 May 2010, at 00:51, Iain Buchanan wrote:
...
... Inotify has two drawbacks which make it hard or even impossible
to use for Iain's use case:
a) It does not work recursively which means that you have to
create a
new handle for each subdirectory. Of course, this only means more
work
for the programmer but there is also the problem that
Pardon me. I assumed that "files inside the directory" meant that foo
would be be changed when foo/bar changed, thus monitoring grunt would
reflect changes in grunt/foo/bar. I overlooked that a directory is
not
a file.
isn't it? I thought a directory was just a file containing names (or
inodes) of other files? Which would explain why monitoring grunt
wouldn't show changes in grunt/foo/bar, since the directory/file
called
grunt remains the same (ie. contains the same list of inodes) even if
grunt/foo/bar changes.
But if bar changes, the modification time of foo changes, which is a
change in grunt. Apparently not. :(
Let me tell you what I actually want to do, which I may not have made
clear originally:
I want to backup root to an external drive (or that could be rephrased
as "I want to backup any mount to any other mount), such that:
1. My backup is an hour or so out of date (at most)
2. I don't need to copy the entire filesystem every time
To do that, I could either:
* Run rsync every hour over the entire filesystem (I'm doing this
now with ionice, takes about 10 minutes when there are no
changes)
* Use some file notification monitor to tell me which file was
just changed, and only rsync that file
The problems with rsync is that during the rsync process, the
filesystem
is changing, so I will end up with a slightly inconsistent backup.
I really appreciate where you're coming from, as I have the same
concerns regarding the backing up of emails. I have systems with gigs
of email on them - belonging to myself and to others - and in the time
taken to rsync the lot, more mail is sure to have arrived. All one can
do is rsync regularly - then you're only an hour or so out of date.
I'm considering LVM for it's snapshot capability, but I'd still have
to
rsync root.
Indeed, snapshotting is ideal.
I would prefer a file notification method as well, so I can
just rsync the file that just changed.
I think you're unrealistic to expect your backup to be uptodate to
within minutes or seconds. I don't really see the benefit of that, to
be honest. As far as I'm concerned I don't really care if /bin, /sbin
&c are backed up or not, because I can reinstall Gentoo in a couple of
hours. Here /etc does not generally change on a daily basis - more
like a backup from last month would be fine! Your milage may vary, but
I really don't see a day-old backup of /etc as a problem. So we are
really only down to /home which needs backing up as frequently as
hourly.
I assume that you're aware that "live" backups require multiple
copies. This is the RAID-is-not-a-backup argument - if a file becomes
corrupt you don't want it overwriting your backup copy.
Having said that you're "unrealistic" and that "I don't see the
benefit", I do appreciate where you're coming from. It's the answer
that us perfectionists want - it's the answer that all Linux users
want. And thus, if there were a clean and elegant solution then it
would be a well-trodden path. I think that if there was a clean and
elegant solution to "the perfect backup" everyone would know what it
was, and everyone would be using it, and you wouldn't even have to
ask. This is why I stopped reading your thread early.
Having said all that, if the stumbling block is snapshotting of your /
partition, you might take a look at btrfs - it looks like it allows
you to do this. brrfs is probably good enough now for everyday use, as
long as you have a backup, and I think you could reasonably easily
backup your snapshots on some regular basis to some other "safe"
filesystem, such as ext[234].
I find it slightly interesting that none of the ionotify solutions do
what you want. The point of such tools is to "do something when a file
is written or changed", so they would seem an ideal fit to solve the
"perfect backup problem" - in that context it would seem trivial to
write a kernel module / userspace app to write a copy of each changed
file to /path/to/backup/$UID and record some information about the
original filename, $UID and the time of change. I imagine that some
types of changes to files - such as bittorrent downloads, in which a
binary file may change like A___E -> A_C_E -> ABCDE - are more
difficult to handle than others, so this will be left to the reader as
a practical programming exercise.
Stroller.
