Re: True repository backup

[Andreas Stieger]

Александр Козлов wrote:
> The [subversion repository restore] problem is:
> 
> - data is backed-up every night, it's ok, but
> - many (50) users do they commits couple of times every day, and
> - users' working copies are out of sync (they are later!) with server 
> after backup restoration.

Yes, but you knew that when you configured nightly backup.

> Of cause, every user can do a virgin repository checkout and then commit 
> his/her latest data. Many users have some local unversioned data inside 
> ours working copy directories, so they would also synchronize this data.

If you are talking about uncommitted data, that is not even a consideration for 
svn.
 
> Latest data and even file revisions are in clients svn databases. So, my 
> question: is there some "magic" instrument, that helps users or admins 
> to resynchronize working copies and commit latest data into the server?

Working copies are considered throw-away data. Subversion has no facilities to 
re-sync working copies to a restored server repositories that may have some 
revisions removed, in the case where a working copy has newer revisions 
locally. 

> The Red Book says that making backups every hour is a form of paranoia. 
> But after some moments every admin tends to be the real paranoic:)

That has nothing to do with paranoia, just cost vs. benefits. If nightly 
backups with archives covers what you need at a price you want to spend for it, 
then do that. If you need running back-ups and are willing to "pay" for it, 
implement it. Many people fail to protect their invested effort properly 
testing the configuration in a real life drill, a.k.a. test restore.

> will try to use a repository mirror or some distributed version control. 
> But I would teach all users to use the new instruments.

I am not sure what you are saying here as these are different concepts.

svnsync mirrors can bridge the gap between interval, archived backups and the 
intra-day data, but NEVER replace them. As part of a mixed strategy they 
absolutely make sense for your case. They are not synchronous, neither are dvcs 
for that matter, and synchronicity is a different consideration.

Andreas

Re: True repository backup

[Stefan Hett]

Hi,
On 12/1/2016 4:47 PM, Александр Козлов wrote:

Hello!

I use the Subversion a couple of years. I initiated to deploy the 
Subversion to our work server, because its installation and usage are 
quite simple. And I sometimes help users and administrator to work 
with svn.


Everything was very good, and Yours program realy helps us in our 
work. But the moment of server failure always comes, when anyone wait 
them :) And now we have a problem to restore the Subversion 
repositories from a backup. I read many posts in www about repository 
backup saving, but practically there are no information about real 
backup restoration experience. The our problem is:


- data is backed-up every night, it's ok, but
- many (50) users do they commits couple of times every day, and
- users' working copies are out of sync (they are later!) with server 
after backup restoration.


Unfortunately, the Subersion client programs don't know what to do, 
when working copy base revision is later then head one. I tried to do 
a dummy commits to increase repository head revision number, and then 
tried to coomit the latest version - but this wasn't help to 
synchronize working copies with server.


Of cause, every user can do a virgin repository checkout and then 
commit his/her latest data. Many users have some local unversioned 
data inside ours working copy directories, so they would also 
synchronize this data. But the main problem - working copies have 
different base revisions. And even files and directories are 
different. It is difficult to find the latest working copy of each file.


Latest data and even file revisions are in clients svn databases. So, 
my question: is there some "magic" instrument, that helps users or 
admins to resynchronize working copies and commit latest data into the 
server?


The Red Book says that making backups every hour is a form of 
paranoia. But after some moments every admin tends to be the real 
paranoic:) I will try to use a repository mirror or some distributed 
version control. But I would teach all users to use the new instruments.



An idea to restore the missing data for your users would be to:
1. Restore the existing backup on a repository.
2. Have your users do a fresh checkout of the repository.
3. Copy over all files from their previous working copy to the fresh one.
4. Commit all changes which were missing from the backup.

That way you can reduce the work which was lost to a minimum, I guess.

I won't go into the other areas, since Andreas already answered these.

--
Regards,
Stefan Hett

Re: True repository backup

[Jörg Heßdörfer]

Hi,

On 02.12.2016 10:06, Stefan Hett wrote:

Hi,
On 12/1/2016 4:47 PM, Александр Козлов wrote:

Hello!

I use the Subversion a couple of years. I initiated to deploy the 
Subversion to our work server, because its installation and usage are 
quite simple. And I sometimes help users and administrator to work 
with svn.


Everything was very good, and Yours program realy helps us in our 
work. But the moment of server failure always comes, when anyone wait 
them :) And now we have a problem to restore the Subversion 
repositories from a backup. I read many posts in www about repository 
backup saving, but practically there are no information about real 
backup restoration experience. The our problem is:


- data is backed-up every night, it's ok, but
- many (50) users do they commits couple of times every day, and
- users' working copies are out of sync (they are later!) with server 
after backup restoration.



[...]

what we do here is using post-commit hook scripts on the server to dump 
the commit data to a queue folder and transfer these files periodically 
(every 5 mins) to our backup system. Additionally, all changed repos are 
dumped nightly. Thus, we have near realtime recovery possibility 
whenever desaster strikes (which it never did in over 10 years now, 
probably because of our backup/recovery strategy ;-)).


Best regards,
Jörg
--

Jörg Heßdörfer
S.E.A. Datentechnik GmbH
Mülheimer Straße 7
53840 Troisdorf
Tel.: +49 2241 12737-19
Fax.: +49 2241 12737-14
E-Mail: joerg.hessdoer...@sea-gmbh.com
Web   : http://www.sea-gmbh.com

Informationen nach 37aHGB, 35a GmbH-Gesetz:
S.E.A.
Science&  Engineering Applications
Datentechnik GmbH
Sitz der Gesellschaft Köln
Handelsregister Köln HRB 27016
Geschäftsführer: Dr. Gerd Schmitz, Wolfram Koerver

Re: True repository backup

[Александр Козлов]

Hello, Andreas!

Great thanks for Your fast reply!

02.12.2016 11:43, Andreas Stieger пишет:

Александр Козлов wrote:
Many people fail to protect their invested effort properly testing the 
configuration in a real life drill, a.k.a. test restore.


Yes, I absolutely agree with You. In our case the restore testing was 
not done.



svnsync mirrors can bridge the gap between interval, archived backups and the 
intra-day data, but NEVER replace them. As part of a mixed strategy they 
absolutely make sense for your case. They are not synchronous, neither are dvcs 
for that matter, and synchronicity is a different consideration.


Ok. We will try svnsync.

--
Best regards!

Alexander Kozlov
Senior hardware developer
Saint-Petersburg
mailto:koz...@avionica.spb.ru

Feature Request: svn upgrade [--check][--check-current]

[Marc Pawlowsky]

There is a lot of effort and hacks on determining what SVN database
version is being used on the working copy.  Usually resorting to
examining the internal contents of a .svn/* file.

For future releases it would be nice if there was a command to display
if the current working copy is at the same level as the svn version
being used, and if not if it is upgradable.

The use case is to be able to detect early on in a build process if
the wrong version of SVN was used.

Examples:
svn upgrade --check .
CURRENT: yes
UPGRADABLE: yes
VERSION: 20
exit code 0 to indicate upgradable, which should be a no-op

svn upgrade --check .
CURRENT: no
UPGRADABLE: yes
VERSION: 19
exit code 0 to indicate upgradable

svn upgrade --check .
CURRENT: no
UPGRADABLE: no
VERSION: unknown
exit code non-zero to indicate not upgradable

svn upgrade --check-current .
CURRENT: yes
UPGRADABLE: yes
VERSION: 20
exit code 0 to indicate no upgrade is needed

svn upgrade --check-current .
CURRENT: NO
UPGRADABLE: yes
VERSION: 19
exit code 1 to indicate  upgrade is needed

svn upgrade --check-current .
CURRENT: NO
UPGRADABLE: no
VERSION: unknown
exit code 2 to indicate  upgrade cannot be performed

Re: Feature Request: svn upgrade [--check][--check-current]

[Daniel Shahaf]

Marc Pawlowsky wrote on Fri, Dec 02, 2016 at 11:50:14 -0500:
> There is a lot of effort and hacks on determining what SVN database
> version is being used on the working copy.  Usually resorting to
> examining the internal contents of a .svn/* file.

The incantation is:

format_number=`head -n1 .svn/format`
if [ "$format_number" -eq 12 ]; then
  format_number=`sqlite3 .svn/wc.db 'pragma user_version;'`
fi

This inspects .svn directly, and as such is *not* guaranteed to be
forward compatible.  (We don't anticipate breaking it, but we may.)

See SVN_WC__VERSION (in wc.h) for the format-number-to-minor-version map.

> For future releases it would be nice if there was a command to display
> if the current working copy is at the same level as the svn version
> being used, and if not if it is upgradable.
> 
> The use case is to be able to detect early on in a build process if
> the wrong version of SVN was used.
> 

Couldn't you run, say, 'svn info >/dev/null' and see whether it gives
error E155036 or not?

% ./tools/dev/which-error.py E155036
00155036  SVN_ERR_WC_UPGRADE_REQUIRED

Going through your scenarios:

> svn upgrade --check-current .
> CURRENT: yes
> UPGRADABLE: yes
> VERSION: 20
> exit code 0 to indicate no upgrade is needed

'svn info' will exit 0.

> svn upgrade --check-current .
> CURRENT: NO
> UPGRADABLE: yes
> VERSION: 19
> exit code 1 to indicate  upgrade is needed

'svn info' will report E155036 (and 'svn upgrade' will succeed).

> svn upgrade --check-current .
> CURRENT: NO
> UPGRADABLE: no
> VERSION: unknown
> exit code 2 to indicate  upgrade cannot be performed

Only happens in corner cases (e.g.,
).
'svn info' will report E155036 and 'svn upgrade' will error out.

I'm not sure whether we have today a way to distinguish "non-current,
non-upgradeable" from "non-current, upgradeable", without actually
running 'svn upgrade'.  Perhaps others can speak to that.

>

To be clear, I'm not opposed to adding such functionality; I'm simply
trying to understand why the use-case can't be addressed with existing
features.

Cheers,

Daniel

Re: Feature Request: svn upgrade [--check][--check-current]

[Marc Pawlowsky]

I had found some of the magic formulas before, which was how I was aware of
looking into the .svn directory.   Thank you for the detailed responses
which I can wrap into local commands, and should be sufficient for my needs.

However I believe  a supported, documented, and forward compatible way is
preferable;  given the volume of questions that come up when googling it
seems to be a popular item.

The problem I am working on is to have a bash script exit early with a
clear error message if:

- the script was checked out in a svn workspace version that is not
expected, older or newer.
- the script is executed on a host that has an old version of svn that
cannot work with the workspace.
- the script is executed  on a host that has a newer version of svn, such
that the workspace would have to be upgraded.

It is an issue when there are manual operations in a distributed
environment with  a shared file system.  It is a common error to
accidentally perform manual operations on a host that has the wrong svn
version.  Especially since the old versions have to be retained on some
workstations.

Thanks for the quick response

Marc

On Dec 2, 2016 12:13 PM, "Daniel Shahaf"  wrote:

> Marc Pawlowsky wrote on Fri, Dec 02, 2016 at 11:50:14 -0500:
> > There is a lot of effort and hacks on determining what SVN database
> > version is being used on the working copy.  Usually resorting to
> > examining the internal contents of a .svn/* file.
>
> The incantation is:
>
> format_number=`head -n1 .svn/format`
> if [ "$format_number" -eq 12 ]; then
>   format_number=`sqlite3 .svn/wc.db 'pragma user_version;'`
> fi
>
> This inspects .svn directly, and as such is *not* guaranteed to be
> forward compatible.  (We don't anticipate breaking it, but we may.)
>
> See SVN_WC__VERSION (in wc.h) for the format-number-to-minor-version map.
>
> > For future releases it would be nice if there was a command to display
> > if the current working copy is at the same level as the svn version
> > being used, and if not if it is upgradable.
> >
> > The use case is to be able to detect early on in a build process if
> > the wrong version of SVN was used.
> >
>
> Couldn't you run, say, 'svn info >/dev/null' and see whether it gives
> error E155036 or not?
>
> % ./tools/dev/which-error.py E155036
> 00155036  SVN_ERR_WC_UPGRADE_REQUIRED
>
> Going through your scenarios:
>
> > svn upgrade --check-current .
> > CURRENT: yes
> > UPGRADABLE: yes
> > VERSION: 20
> > exit code 0 to indicate no upgrade is needed
>
> 'svn info' will exit 0.
>
> > svn upgrade --check-current .
> > CURRENT: NO
> > UPGRADABLE: yes
> > VERSION: 19
> > exit code 1 to indicate  upgrade is needed
>
> 'svn info' will report E155036 (and 'svn upgrade' will succeed).
>
> > svn upgrade --check-current .
> > CURRENT: NO
> > UPGRADABLE: no
> > VERSION: unknown
> > exit code 2 to indicate  upgrade cannot be performed
>
> Only happens in corner cases (e.g.,
> ).
> 'svn info' will report E155036 and 'svn upgrade' will error out.
>
> I'm not sure whether we have today a way to distinguish "non-current,
> non-upgradeable" from "non-current, upgradeable", without actually
> running 'svn upgrade'.  Perhaps others can speak to that.
>
> >
>
> To be clear, I'm not opposed to adding such functionality; I'm simply
> trying to understand why the use-case can't be addressed with existing
> features.
>
> Cheers,
>
> Daniel
>