Re: Problems with making hardlink-based backups

2009-08-19 Thread David
On Tue, Aug 18, 2009 at 5:11 PM, Andrew Sackville-West wrote: > I've had rdiff-backup fail because of mis-matched versions. Again, not > to belabor the obvious, but do you have compatible versions of > rdiff-backup on each machine? If you have compatible (i.e., the same) > versions on both ends and

Re: Problems with making hardlink-based backups

2009-08-18 Thread Andrew Sackville-West
On Tue, Aug 18, 2009 at 03:11:47PM +0200, David wrote: [...] > Basically, rdiff-backup was perfect for a while. But then we upgraded > the server to Lenny. And then it stopped working T_T. I think that > rdiff-backup's author must have changed something, which now causes > huge ram usage for large

Re: Problems with making hardlink-based backups

2009-08-18 Thread David
On Mon, Aug 17, 2009 at 6:26 PM, Andrew Sackville-West wrote: > Here's another question: what is stored in all these millions of > files? > [...] Basically, the lions share would be tonnes of user-generated files, for example huge numbers of image files (and thumbnails) that get stored in director

Re: Problems with making hardlink-based backups

2009-08-17 Thread Andrew Sackville-West
On Mon, Aug 17, 2009 at 10:59:20AM +0200, David wrote: > Thanks for the replies. [...] > > Basically, the problem isn't that I don't know how to use rsync, cp, > etc to make the backups, manage generations, etc... the problem is an > incredibly large filesystem (as in number of hardlinks, and act

Re: Problems with making hardlink-based backups

2009-08-17 Thread David
Err.. and another post on backuppc, sorry. I think that backuppc is actually going to have the same problem (with massive filesystems causing du and locate, etc to become next to unusable for the backup storage directories). The reason for this: "Therefore, every file in the pool will have at lea

Re: Problems with making hardlink-based backups

2009-08-17 Thread David
Sorry for spamming the list.. I think I didn't read the docs correctly, before posting the above. It seems that backuppc actully does keep recent snapshots that aren't in the pool... so scripts, admins, etc can get to them easily without going through backuppc scripts. It looks like backuppc actu

Re: Problems with making hardlink-based backups

2009-08-17 Thread David
On Sat, Aug 15, 2009 at 4:35 AM, Rob Owens wrote: > You might want to check out BackupPC.  It uses hardlinks and > compression.  It's got a web-based GUI which makes it pretty easy to > find statistics on disk space used, on a per-server basis. > I've been researching backuppc, and it seems like i

Re: Problems with making hardlink-based backups

2009-08-17 Thread David
Thanks for the replies. On Fri, Aug 14, 2009 at 5:05 PM, Andrew Sackville-West wrote: >> >> du worked pretty well with rdiff-backup, but is very problematic with >> a large number of hardlink-based snapshots, which each have a complete >> "copy" of a massive filesystem (rather than just info on wh

Re: Problems with making hardlink-based backups

2009-08-14 Thread Rob Owens
On Thu, Aug 13, 2009 at 09:20:17AM +0200, David wrote: > Hi list. > > Until recently I was using rdiff-backup for backing up our servers. > But after a certain point, the rdiff-backup process would take days, > use up huge amounts of CPU and RAM, and the debug logs were very > opaque. And the rdif

Re: Problems with making hardlink-based backups

2009-08-14 Thread Alan Chandler
Andrew Sackville-West wrote: On Fri, Aug 14, 2009 at 08:43:32AM +0200, David wrote: Thanks for your suggestion, and I have heard of rsnapshot. Although, actually removing older snapshot directories isn't really the problem. The problem is, if you have a large number of such backups (perhaps on

Re: Problems with making hardlink-based backups

2009-08-14 Thread Andrew Sackville-West
On Fri, Aug 14, 2009 at 08:43:32AM +0200, David wrote: > Thanks for your suggestion, and I have heard of rsnapshot. > > Although, actually removing older snapshot directories isn't really the > problem. > > The problem is, if you have a large number of such backups (perhaps > one per server), th

Re: Problems with making hardlink-based backups

2009-08-13 Thread David
Btw, sorry for top-posting. I don't use mailing lists very often and forgot about the convention. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Re: Problems with making hardlink-based backups

2009-08-13 Thread David
Thanks for your suggestion, and I have heard of rsnapshot. Although, actually removing older snapshot directories isn't really the problem. The problem is, if you have a large number of such backups (perhaps one per server), then finding out where harddrive space is actually being used, is proble

Re: Problems with making hardlink-based backups

2009-08-13 Thread Andrew Sackville-West
On Thu, Aug 13, 2009 at 09:20:17AM +0200, David wrote: > Hi list. [...] > 3) Existing tools for managing hardlink-based snapshot directories > etc. maybe rsnapshot is what you're after. It does hardlinked snapshots with automagical deletion of older backups and configurable frequency etc. I quite

Problems with making hardlink-based backups

2009-08-13 Thread David
Hi list. Until recently I was using rdiff-backup for backing up our servers. But after a certain point, the rdiff-backup process would take days, use up huge amounts of CPU and RAM, and the debug logs were very opaque. And the rdiff-backup mailing lists weren't very helpful either. So, for server