How to safely back up an svn repository on ubuntu?

[Bo Berglund]

I would like to set up a cron job to backup the repositories on an svn server to
a Synology NAS.

The repositories reside in /var/lib/svn and comprise about 6 Gb worth of files
in 12 different directories.
These are svnsync:ed nightly from our main SVN server in the main office across
the world as a backup of the development activities there.
The main svn server is VisualSvn on Windows Server 16 if that matters. The
svnsync has worked fine for 2+ years since I set it up.

The backup SVN server runs on an Ubuntu 18.4 LTS server in my home network and
on that network I also have a Synology NAS, which I want to use as a 3rd level
backup for the SVN repositories in case the Ubuntu server here crashes.

I got a problem with it a few days ago when we had a power outage and the Ubuntu
box did not boot properly when power returned. Now fixed but it raised the
concern over backup security.

I have now set up an NFS connection between the Ubuntu server and the Synology
NAS so I have a target dir in Ubuntu in /nfs/backup where I can write the files.

So what I had in mind is to somehow keep a copy of the repository files on the
nfs share but I am not sure what command would be suitable to perform this
action.

I could use tar but that would be complex since there are certainly not many
files that change from one day to the other. Only changed files should be copied
to reduce the time of backup.
Or else perhaps rsync, provided it knows which files have changed and which have
not etc.

I am a user of Ubuntu Linux for some of my work and have had to learn the basics
of managing Linux, but I am not versed in these actions. Daily I am a Windows
user.

I don't know if some actions should be done towards svn on the ubuntu server in
order to safely copy the files either...

Please advice...


-- 
Bo Berglund
Developer in Sweden

Re: How to safely back up an svn repository on ubuntu?

[Daniel Sahlberg]

Den ons 13 jan. 2021 kl 12:16 skrev Bo Berglund :

> Please advice...
>

You can probably get started by taking a look at the Repository Backup
section of the SVN Book
http://svnbook.red-bean.com/nightly/en/svn.reposadmin.maint.html#svn.reposadmin.maint.backup

//Daniel

Re: How to safely back up an svn repository on ubuntu?

[Mark Phippard]

On Wed, Jan 13, 2021 at 6:16 AM Bo Berglund  wrote:

> I would like to set up a cron job to backup the repositories on an svn
> server to
> a Synology NAS.
>
> The repositories reside in /var/lib/svn and comprise about 6 Gb worth of
> files
> in 12 different directories.
> These are svnsync:ed nightly from our main SVN server in the main office
> across
> the world as a backup of the development activities there.
> The main svn server is VisualSvn on Windows Server 16 if that matters. The
> svnsync has worked fine for 2+ years since I set it up.
>
> The backup SVN server runs on an Ubuntu 18.4 LTS server in my home network
> and
> on that network I also have a Synology NAS, which I want to use as a 3rd
> level
> backup for the SVN repositories in case the Ubuntu server here crashes.
>
> I got a problem with it a few days ago when we had a power outage and the
> Ubuntu
> box did not boot properly when power returned. Now fixed but it raised the
> concern over backup security.
>
> I have now set up an NFS connection between the Ubuntu server and the
> Synology
> NAS so I have a target dir in Ubuntu in /nfs/backup where I can write the
> files.
>
> So what I had in mind is to somehow keep a copy of the repository files on
> the
> nfs share but I am not sure what command would be suitable to perform this
> action.
>
> I could use tar but that would be complex since there are certainly not
> many
> files that change from one day to the other. Only changed files should be
> copied
> to reduce the time of backup.
> Or else perhaps rsync, provided it knows which files have changed and
> which have
> not etc.
>
> I am a user of Ubuntu Linux for some of my work and have had to learn the
> basics
> of managing Linux, but I am not versed in these actions. Daily I am a
> Windows
> user.
>
> I don't know if some actions should be done towards svn on the ubuntu
> server in
> order to safely copy the files either...
>

I would use rsync. Especially since this sounds like it is mainly a last
resort backup. There are some relatively minor problems that can happen if
rsync happens to run in the middle of a transaction.  The rsync still works
but it could make a backup of the repository in an incomplete state. Those
problems are fixable and the next rsync would still work and would fix the
problem too.

The safer way is to use svnadmin hotcopy but you would have to script it
since you need to run it for each repository. The advantage of rsync is
that you can just process the entire root folder where your repositories
are stored.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: How to safely back up an svn repository on ubuntu?

[Pierre Fourès]

Hi Bo,

I understand your point and have done similar setup. I've first started
using a local svnadmin hotcopy to create a clean and safe copy of my
repository, and then, rsynced this folder to my remote server. As I can
handle a downtime on my svn server (in the middle of the night), I've
switched to shutting down the server, and just rsyncing the /var/lib/svn
sub-folders. This saved a step, a loop, and some disk usage on my local
server.

As an extra-layer of security, what you might consider too, on your
cold-backup NAS system, is to set up a rolling backup strategy scheme. I
was afraid of a corruption of my repositories, for which ever reason, but
mostly from a mistake from my own self. Having multiple copies on my cold
storage helps preventing it. Just using the day of the week while selecting
which folder you want to rsync your files too makes you a 7 days rolling
scheme.

Pierre.

Le mer. 13 janv. 2021 à 12:16, Bo Berglund  a écrit :

> I would like to set up a cron job to backup the repositories on an svn
> server to
> a Synology NAS.
>
> The repositories reside in /var/lib/svn and comprise about 6 Gb worth of
> files
> in 12 different directories.
> These are svnsync:ed nightly from our main SVN server in the main office
> across
> the world as a backup of the development activities there.
> The main svn server is VisualSvn on Windows Server 16 if that matters. The
> svnsync has worked fine for 2+ years since I set it up.
>
> The backup SVN server runs on an Ubuntu 18.4 LTS server in my home network
> and
> on that network I also have a Synology NAS, which I want to use as a 3rd
> level
> backup for the SVN repositories in case the Ubuntu server here crashes.
>
> I got a problem with it a few days ago when we had a power outage and the
> Ubuntu
> box did not boot properly when power returned. Now fixed but it raised the
> concern over backup security.
>
> I have now set up an NFS connection between the Ubuntu server and the
> Synology
> NAS so I have a target dir in Ubuntu in /nfs/backup where I can write the
> files.
>
> So what I had in mind is to somehow keep a copy of the repository files on
> the
> nfs share but I am not sure what command would be suitable to perform this
> action.
>
> I could use tar but that would be complex since there are certainly not
> many
> files that change from one day to the other. Only changed files should be
> copied
> to reduce the time of backup.
> Or else perhaps rsync, provided it knows which files have changed and
> which have
> not etc.
>
> I am a user of Ubuntu Linux for some of my work and have had to learn the
> basics
> of managing Linux, but I am not versed in these actions. Daily I am a
> Windows
> user.
>
> I don't know if some actions should be done towards svn on the ubuntu
> server in
> order to safely copy the files either...
>
> Please advice...
>
>
> --
> Bo Berglund
> Developer in Sweden
>
>

RE: How to safely back up an svn repository on ubuntu?

[Juan Gabriel Covas]

If it helps, I use the following one-liner to "svnadmin hotcopy --incremental", 
so only new revisions are copied each time (the svn-hot-copies target path is a 
NFS mount, for example, in other box)

# find /path/to/your/repos -maxdepth 1 -mindepth 1 -type d -printf "%f\0" | 
sort -z | xargs -0 -I % sh -c 'echo %; svnadmin hotcopy --incremental 
/path/to/your/repos/% /path/to/svn-hot-copies/repos/%;'

Regards,

Juan G. Covas
Balearic Islands

De: Mark Phippard  
Enviado el: miércoles, 13 de enero de 2021 12:54
Para: Bo Berglund 
CC: Subversion 
Asunto: Re: How to safely back up an svn repository on ubuntu?

On Wed, Jan 13, 2021 at 6:16 AM Bo Berglund  
wrote:
I would like to set up a cron job to backup the repositories on an svn server to
a Synology NAS.

The repositories reside in /var/lib/svn and comprise about 6 Gb worth of files
in 12 different directories.
These are svnsync:ed nightly from our main SVN server in the main office across
the world as a backup of the development activities there.
The main svn server is VisualSvn on Windows Server 16 if that matters. The
svnsync has worked fine for 2+ years since I set it up.

The backup SVN server runs on an Ubuntu 18.4 LTS server in my home network and
on that network I also have a Synology NAS, which I want to use as a 3rd level
backup for the SVN repositories in case the Ubuntu server here crashes.

I got a problem with it a few days ago when we had a power outage and the Ubuntu
box did not boot properly when power returned. Now fixed but it raised the
concern over backup security.

I have now set up an NFS connection between the Ubuntu server and the Synology
NAS so I have a target dir in Ubuntu in /nfs/backup where I can write the files.

So what I had in mind is to somehow keep a copy of the repository files on the
nfs share but I am not sure what command would be suitable to perform this
action.

I could use tar but that would be complex since there are certainly not many
files that change from one day to the other. Only changed files should be copied
to reduce the time of backup.
Or else perhaps rsync, provided it knows which files have changed and which have
not etc.

I am a user of Ubuntu Linux for some of my work and have had to learn the basics
of managing Linux, but I am not versed in these actions. Daily I am a Windows
user.

I don't know if some actions should be done towards svn on the ubuntu server in
order to safely copy the files either...

I would use rsync. Especially since this sounds like it is mainly a last resort 
backup. There are some relatively minor problems that can happen if rsync 
happens to run in the middle of a transaction.  The rsync still works but it 
could make a backup of the repository in an incomplete state. Those problems 
are fixable and the next rsync would still work and would fix the problem too.

The safer way is to use svnadmin hotcopy but you would have to script it since 
you need to run it for each repository. The advantage of rsync is that you can 
just process the entire root folder where your repositories are stored.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Switching to a specific revision

[Anton Shepelev]

Hello, all

I want to `switch' to a location that existsed in a previous
revision, but has since been moved.  I invoke:

   svn switch -r 1431 ^^/Client/B1/Addons/AddCost/ --ignore-ancestry

and receive:

   svn: E160013: '/svn/Sources/!svn/rvr/6932/Client/B1/Addons/AddCost' path not 
found

This path is indeed not present in r6932, but it exists in
r1431. Why does SVN try to access the later revision in
spite of being told to use 1431?

Re: Switching to a specific revision

[Nathan Hartman]

On Wed, Jan 13, 2021 at 10:59 AM Anton Shepelev  wrote:
> I want to `switch' to a location that existsed in a previous
> revision, but has since been moved.  I invoke:
>
>svn switch -r 1431 ^^/Client/B1/Addons/AddCost/ --ignore-ancestry
>
> and receive:
>
>svn: E160013: '/svn/Sources/!svn/rvr/6932/Client/B1/Addons/AddCost' path 
> not found
>
> This path is indeed not present in r6932, but it exists in
> r1431. Why does SVN try to access the later revision in
> spite of being told to use 1431?

Because '-r' tells SVN to look for that path in the HEAD revision and
then follow it back to the specified revision.

Try using a peg revision instead:

svn switch ^^/Client/B1/Addons/AddCost/@1431 --ignore-ancestry

See: http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html

("Version Control With Subversion," chapter 3 "Advanced Topics,"
section "Peg and Operative Revisions")

Cheers,
Nathan

Re: Switching to a specific revision

[Anton Shepelev]

Nathan Hartman:

> Because '-r' tells SVN to look for that path in the HEAD
> revision and then follow it back to the specified
> revision.
>
> Try using a peg revision instead:
> svn switch ^^/Client/B1/Addons/AddCost/@1431 --ignore-ancestry
>
> See: http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html

Thank you, Nathan. That answers my question. I did not know about
peg revisions.

Re: How to safely back up an svn repository on ubuntu?

[Bo Berglund]

On Wed, 13 Jan 2021 12:58:59 +0100, Pierre Fourès 
wrote:

Thanks for your comments Pierre! :)

>I understand your point and have done similar setup. I've first started
>using a local svnadmin hotcopy to create a clean and safe copy of my
>repository, and then, rsynced this folder to my remote server. As I can
>handle a downtime on my svn server (in the middle of the night), I've
>switched to shutting down the server, and just rsyncing the /var/lib/svn
>sub-folders. This saved a step, a loop, and some disk usage on my local
>server.

So you mean I need to shut down the svn server first?
I don't know how to do that since svn is kind of integrated with apache on my
server...
OTOH I know exactly when the nightly svnsync is started on the source system to
copy over the day's changes to the backup server, so I could choose another time
to run my cron job.
And this backup svn server on Ubuntu is not used for anything on the synced
repositories except receiving the backup data.


Concerning rsync, I have first tested what is happening when I use ordinary file
system commands towards the nfs share on Synology...

What I found was that the owner/group fields look strange to me.
Here an example:

$ ls -l ~/www/index.php
-rw-rw-r--  1 bosse bosse177 Jan 13 21:32 index.php

$ cp ~/www/index.php /nfs/backup/

Then:
$ ls -l /nfs/backup/
-rw-rw-r-- 1 nobody 4294967294  177 Jan 13 23:04 index.php

Notice how the file ownership has changed in this operation.
Is rsync doing something else that keeps the ownership the same as on the
source?
Or have I set up the wrong kind of options on the mount command for the nfs
share?

>As an extra-layer of security, what you might consider too, on your
>cold-backup NAS system, is to set up a rolling backup strategy scheme. I
>was afraid of a corruption of my repositories, for which ever reason, but
>mostly from a mistake from my own self. Having multiple copies on my cold
>storage helps preventing it. Just using the day of the week while selecting
>which folder you want to rsync your files too makes you a 7 days rolling
>scheme.
>

I don't think I will need a rolling scheme here...


-- 
Bo Berglund
Developer in Sweden