----- Original Message ----- > From: "Mark Phippard" <markp...@gmail.com> > To: "amartin" <amar...@xes-inc.com> > Cc: "users" <users@subversion.apache.org> > Sent: Tuesday, December 13, 2016 3:57:04 PM > Subject: Re: Backup using ZFS Snapshots
> On Tue, Dec 13, 2016 at 4:47 PM, Andrew Martin <amar...@xes-inc.com> wrote: > >> >> >> ----- Original Message ----- >> > From: "Mark Phippard" <markp...@gmail.com> >> > To: "amartin" <amar...@xes-inc.com> >> > Cc: "users" <users@subversion.apache.org> >> > Sent: Tuesday, December 13, 2016 3:35:37 PM >> > Subject: Re: Backup using ZFS Snapshots >> >> > On Tue, Dec 13, 2016 at 4:17 PM, Andrew Martin <amar...@xes-inc.com> >> wrote: >> > >> >> Hello, >> >> >> >> I am running a Subversion 1.9.3 server on Ubuntu 16.04. I currently use >> >> svnadmin hotcopy to safely backup the SVN repositories, but since the >> >> repositories are now hosted on a ZFS dataset, I would like to utilize >> ZFS's >> >> snapshot capabilities to create atomic, point-in-time backups of the >> >> repositories. My plan to do this is as follows: >> >> >> >> 1. create zfs snapshot >> >> 2. clone zfs snapshot and mount at a temporary location >> >> 3. run svnadmin hotcopy from the mounted clone to safely create a backup >> >> 4. umount and destroy clone >> >> >> >> My only concern is if a commit is in-progress when the zfs snapshot >> occurs, >> >> would svnadmin hotcopy still be able to safely handle creating the >> backup? >> >> >> > >> > svnadmin hotcopy will not have a problem with an in-process commit. That >> > is kind of one of the points of that command. >> > >> > >> > >> >> Is this a safe procedure for creating backups? >> >> >> > >> > I know very little about ZFS but I do not understand why you would do >> > this. If it can take snapshots, then why wouldn't you just take a >> snapshot >> > of the actual live repository, why would you want to copy it? >> > >> > Setting that aside, I think hotcopy is a very good way to do backups ... >> > but it is not clear what value would come from taking a snapshot of the >> > backup given that version control history is immutable etc. Also, once >> you >> > create the initial hotcopy, you can use the hotcopy --incremental option >> to >> > just copy the newer revisions. Do that from a post-commit hook and you >> can >> > always have a backup up to the latest commit. >> > >> > To me snapshots would seem like a way to do backups without using >> hotcopy. >> > I am not sure the value of combining the two ... but it should work >> > technically if you have "reasons". >> > >> > Mark >> >> The reason I want to do it this way is that I want to "pull" the backups to >> the backup server rather than "pushing" them from the Subversion server. In >> order to do that, I plan on mounting the clone over NFS on the backup >> server >> and then using "svnadmin hotcopy" to pull any new data onto the backup >> server. >> Yes, I could just mount the live repository, but I have a number of >> repositories >> to backup and using the ZFS snapshot ensures that they are all from a >> single >> point-in-time. >> >> It sounds like "svnadmin hotcopy" will just ignore any in-progress commit, >> so >> even if the ZFS snapshot happened in the middle of a commit, it would still >> result in a consistent backup. >> >> > Yes, hotcopy will leave you with a consistent repository at the state it > was in when the backup started. Any commits that are in process will not > be copied/included in the backup. The problem with hotcopy, is that unless > you craft a method where you can use --incremental, you are copying the > entire repository every time you do a backup even though only a small > number of new files may have been created. The SVN repository format is > write-once. Only new files get added as new revisions get created. So the > repository can be backed up very efficiently if you just keep capturing the > new revisions being created. > Yes I plan on using --incremental to make the backup very quick. Basically the steps will be: 1. make snapshot on fileserver 2. clone snapshot to read-only clone, accessible by backup server over NFS 3. mount clone on backup server 4. run svnadmin hotcopy --incremental on clone 5. unmount and delete clone Since I'll be pointing "svnadmin hotcopy" at the same target directory on the backup server each time this is run, I should be able to to take advantage of --incremental to make this fast. > If all of your activity happens via Apache there might be easier ways to make > all your repositories read only during a backup window and you can also always > use the start-commit hook as an easy way to make repositories read only. It's tempting to just stop apache during the backup, but I need to continue to provide read-only access during the backup window, so apache needs to stay on. Yes I could update the authz file and change all entries from rw to r, but that seems trickier given that I have a large authz file.
