Re: Keyword expansion on Unicode text files

[Ulrich Eckhardt]

On Thursday 14 January 2010, Ryan Schmidt wrote:
> On Jan 14, 2010, at 13:15, Harald-René Flasch wrote:
> > what should I do in order to make keyword expansion on Unicode text files
> > working (e.g. .inf files)?
> >
> > - BOM is FF FE (used Notepad Save As --> Unicode to create the file)
> > - There is only the svn:keywords property on the file
>
> Sorry, AFAIK there still isn't a way to make svn:keywords work in UTF-16
> files. 

That said, you might be able to store them as UTF-8, which is another full 
Unicode-capable format and even more common than the used UTF-16. Using 
UTF-8, keywork expansion simply works.

Uli

-- 
ML: http://subversion.tigris.org/mailing-list-guidelines.html
FAQ: http://subversion.tigris.org/faq.html
Docs: http://svnbook.red-bean.com/

Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

**
Sator Laser GmbH, Fangdieckstraße 75a, 22547 Hamburg, Deutschland
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
**
   Visit our website at 
**
Diese E-Mail einschließlich sämtlicher Anhänge ist nur für den Adressaten 
bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen 
Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empfänger sein 
sollten. Die E-Mail ist in diesem Fall zu löschen und darf weder gelesen, 
weitergeleitet, veröffentlicht oder anderweitig benutzt werden.
E-Mails können durch Dritte gelesen werden und Viren sowie nichtautorisierte 
Änderungen enthalten. Sator Laser GmbH ist für diese Folgen nicht 
verantwortlich.
**

way of feature request

[Claudius Sailer]

Hi,

what is the way or what is the best way, when I want to add a feature
request?

What I am looking for.

IMHO I can find in OS Unix, Linux, Windows, MacOS X for every file the
"creation date", "modification date" and "last touch date" and I want
(feature request) that these informations are stored in the repository for
every file. And I want to have the option that I can configure that this
date informations are rebuild for every file when I checkout, update,
import, merge or whatever in a repository.


bye


Claudius

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

RE: Keyword expansion on Unicode text files

[Harald-René Flasch]

Thank you for your response!

It shouldn't be a unresolvable programming issue to support keyword expansion 
for Unicode text files?

--hfrmobile









> Subject: Re: Keyword expansion on Unicode text files
> From: subversion-20...@ryandesign.com
> Date: Thu, 14 Jan 2010 16:02:32 -0600
> CC: users@subversion.apache.org
> To: harald-rene.fla...@hotmail.com
> 
> 
> On Jan 14, 2010, at 13:15, Harald-René Flasch wrote:
> 
> > what should I do in order to make keyword expansion on Unicode text files 
> > working (e.g. .inf files)?
> > 
> > - BOM is FF FE (used Notepad Save As --> Unicode to create the file)
> > - There is only the svn:keywords property on the file
> 
> Sorry, AFAIK there still isn't a way to make svn:keywords work in UTF-16 
> files. See:
> 
> http://svn.haxx.se/users/archive-2005-11/0538.shtml
> 
> Possibly related tickets:
> 
> http://subversion.tigris.org/issues/show_bug.cgi?id=2194
> 
> http://subversion.tigris.org/issues/show_bug.cgi?id=2332
> 
> 
> 
  
_
Das neue Windows 7: Vereinfachen Sie Ihre täglichen Aufgaben. Finden Sie den 
richtigen PC.
http://windows.microsoft.com/shop

Re: way of feature request

[Stefan Sperling]

On Fri, Jan 15, 2010 at 09:25:56AM +0100, Claudius Sailer wrote:
> 
> Hi,
> 
> what is the way or what is the best way, when I want to add a feature
> request?
> 
> What I am looking for.
> 
> IMHO I can find in OS Unix, Linux, Windows, MacOS X for every file the
> "creation date", "modification date" and "last touch date" and I want
> (feature request) that these informations are stored in the repository for
> every file. And I want to have the option that I can configure that this
> date informations are rebuild for every file when I checkout, update,
> import, merge or whatever in a repository.

This has already been requested. See:
http://subversion.tigris.org/issues/show_bug.cgi?id=1256

Thanks,
Stefan

Antwort: Re: way of feature request

[Claudius Sailer]

Hi,

I thought this is only for ModificationDate ;-))


by

Claudius



Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart

|>
||
||
||
||
|  Stefan Sperling   |
| |
||
|  15.01.2010 10:30  |
|>
  
>---|
  | 
  |
  | 
  |
  | 
An|
  |  Claudius Sailer   
  |
  | 
 Kopie|
  |  users@subversion.apache.org
  |
  | 
 Thema|
  |  Re: way of feature request 
  |
  | 
  |
  | 
  |
  | 
  |
  | 
  |
  | 
  |
  
>---|




On Fri, Jan 15, 2010 at 09:25:56AM +0100, Claudius Sailer wrote:
>
> Hi,
>
> what is the way or what is the best way, when I want to add a feature
> request?
>
> What I am looking for.
>
> IMHO I can find in OS Unix, Linux, Windows, MacOS X for every file the
> "creation date", "modification date" and "last touch date" and I want
> (feature request) that these informations are stored in the repository
for
> every file. And I want to have the option that I can configure that this
> date informations are rebuild for every file when I checkout, update,
> import, merge or whatever in a repository.

This has already been requested. See:
http://subversion.tigris.org/issues/show_bug.cgi?id=1256

Thanks,
Stefan
--

LBBW-Info IT-Security:
Die Nachricht war weder verschluesselt noch digital unterschrieben.

RE: sync bug -> corrupted proxy repo

[Jon Foster]

Hi,

Ryan Schmidt wrote:
> But Subversion blocks the commit until the post-commit is done.

That particular SVN client will be blocked.  But if you have
two users committing at the same time, or if a user runs "svn"
twice in parallel, then the post-commit hook will be run in
parallel.

Here's how I tested this.  I created a new repository with
a post-commit hook that takes 30 seconds to run.  I then
checked that it works, and that a normal commit took 30
seconds.  I then did two commits in parallel, and that took
30 seconds.  This shows that the post-commit hook is
running in parallel - if it had been run in series, then
it would have taken 60 seconds for 2 commits.  (I also
checked the output of "ps" and observed the two
"post-commit" processes running).

~$ mkdir svnscratch
~$ cd svnscratch/
~/svnscratch$ svn --version | head -n1
svn, version 1.6.8 (dev build)
~/svnscratch$ svnadmin create repo
~/svnscratch$ cat >repo/hooks/post-commit
#! /bin/bash
sleep 30 
~/svnscratch$ chmod a+x repo/hooks/post-commit
~/svnscratch$ time repo/hooks/post-commit

real0m30.004s
user0m0.000s
sys 0m0.008s
~/svnscratch$ time svn mkdir -m "Test" file://`pwd`/repo/trunk

Committed revision 1.

real0m30.030s
user0m0.008s
sys 0m0.008s
~/svnscratch$ time ( svn mkdir -m "Test" file://`pwd`/repo/branches &
svn mkdir -m "Test" file://`pwd`/repo/tags )

Committed revision 2.
Committed revision 3.

real0m30.069s
user0m0.004s
sys 0m0.020s
~/svnscratch$ 


Kind regards,

Jon


**
This email and its attachments may be confidential and are intended solely for 
the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those 
of Cabot Communications Ltd.

If you are not the intended recipient of this email and its attachments, you 
must take no action based upon them, nor must you copy or show them to anyone.

Cabot Communications Limited
Verona House, Filwood Road, Bristol BS16 3RY, UK
+44 (0) 1179584232

Co. Registered in England number 02817269

Please contact the sender if you believe you have received this email in error.

**


__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
__

Re: sync bug -> corrupted proxy repo

[Johan Corveleyn]

On Fri, Jan 15, 2010 at 12:08 PM, Jon Foster  wrote:
> Hi,
>
> Ryan Schmidt wrote:
>> But Subversion blocks the commit until the post-commit is done.
>
> That particular SVN client will be blocked.  But if you have
> two users committing at the same time, or if a user runs "svn"
> twice in parallel, then the post-commit hook will be run in
> parallel.
>
> Here's how I tested this.  I created a new repository with
> a post-commit hook that takes 30 seconds to run.  I then
> checked that it works, and that a normal commit took 30
> seconds.  I then did two commits in parallel, and that took
> 30 seconds.  This shows that the post-commit hook is
> running in parallel - if it had been run in series, then
> it would have taken 60 seconds for 2 commits.  (I also
> checked the output of "ps" and observed the two
> "post-commit" processes running).

Also, I'm pretty sure that, while the post-commit hook is running for
a particular commit, the commit itself is already visible to other
users. So, as you would expect from the name "*post*-commit hook", the
commit itself is already finalized before the post-commit hook starts
running. Otherwise, people wouldn't be able to do things like
automatically updating a working copy on the server, from within their
post-commit hook. The only thing that has to wait on the post-commit
hook is that particular svn client that's running the commit (as Jon
pointed out).

Regards,
Johan

Re: Subversion queries hanging, timing out

[Andrey Repin]

Greetings, Dave Purrington!

> Lately we have been experiencing intermittent timeouts with our Subversion
> operations. It does not happen initially, but after a while it starts
> happening. Restarting Apache alleviates the problem, but it comes back after
> a time. As you can imagine, this wreaks havoc.

> Our operating environment:

>- server - Windows 2003
>- Apache 2.2.13
>- Subversion server 1.6.3
>- Subversion client 1.6.6
>- mod_auth_sspi 1.0.4-2.0.58
>- 200+ very active users, ~74K files

Server: Apache/2.2.11 (Win32) mod_auth_sspi/1.0.5 SVN/1.6.1 PHP/5.2.2 DAV/2
http://www.nosq.com/blog/2008/06/fixing-mod_auth_sspi-and-ie-losing-post-data/
http://dev.nosq.com/downloads/mod_auth_sspi/mod_auth_sspi_1.0.5b-vc9-2.2.11.zip


--
WBR,
 Andrey Repin (anrdae...@freemail.ru) 15.01.2010, <17:48>

Sorry for my terrible english...

Re: Subversion queries hanging, timing out

[Dave Purrington]

Kudos! The module needs some TLC, I'm sure people will appreciate the work.

Unfortunately, we've moved away from it and will not be moving back. It's
too disruptive to my repo user community to keep changing our auth strategy.

I wish you well!

-dave

On Fri, Jan 15, 2010 at 9:50 AM, Andrey Repin  wrote:

> Greetings, Dave Purrington!
>
> > Lately we have been experiencing intermittent timeouts with our
> Subversion
> > operations. It does not happen initially, but after a while it starts
> > happening. Restarting Apache alleviates the problem, but it comes back
> after
> > a time. As you can imagine, this wreaks havoc.
>
> > Our operating environment:
>
> >- server - Windows 2003
> >- Apache 2.2.13
> >- Subversion server 1.6.3
> >- Subversion client 1.6.6
> >- mod_auth_sspi 1.0.4-2.0.58
> >- 200+ very active users, ~74K files
>
> Server: Apache/2.2.11 (Win32) mod_auth_sspi/1.0.5 SVN/1.6.1 PHP/5.2.2 DAV/2
>
> http://www.nosq.com/blog/2008/06/fixing-mod_auth_sspi-and-ie-losing-post-data/
>
> http://dev.nosq.com/downloads/mod_auth_sspi/mod_auth_sspi_1.0.5b-vc9-2.2.11.zip
>
>
> --
> WBR,
>  Andrey Repin (anrdae...@freemail.ru) 15.01.2010, <17:48>
>
> Sorry for my terrible english...
>
>

svndumpfilter bug in 1.6.6 ?

[Joakim Tjernlund]

I am unable to get svndumpfilter to recognize the dir /tags/tuappl02a-r12a-3, 
doing:
svnadmin --quiet  dump /usr/local/src/fpga-repo/TM-3000 | svndumpfilter 
--drop-empty-revs --skip-missing-merge-sources --renumber-revs --quiet include 
tags/tuappl02a-r12a-3 | grep tuappl02a-r12a-3

yields nothing.

I have double checked this tag in my repo many times and it is there.
Can anyone reproduce this?

I am on gentoo Linux

 Jocke

How to use GNOME keyring with Subversion

[Giulio Troccoli]

Although I have been using Subversion since 1.2, so for quite a while, I'm very 
new to GNOME keyring. I would like to set it up but I don't seem able to do 
that.

I have, for testing, created a VM with CentOS 5.4. Then I have built Subversion 
1.6.6 from source with the --with-gnome-keyring option.

The gnome-keyring-daemon is running (does it need any parameters?) and the 
keyring manager show an unlocked session called "session".

I have change the config file to use gnome-keyring and the servers files with 
store-passwords = yes and store-plaintext-passwords = no.

Still, when I update an existing wc I am asked for the Subversion password 
every time.

As I said I am totally new to keyring and stuff like that, so if anyone has 
successfully set it up please help me :-)

Of course, if you need more information just ask

Thanks
Giulio


Linedata Services (UK) Ltd
Registered Office: Bishopsgate Court, 4-12 Norton Folgate, London, E1 6DB
Registered in England and Wales No 3027851VAT Reg No 778499447

Post-commit hook recipes?

[Peter Ruprecht]

Hi,

I've been looking for a collection of example post-commit hook scripts 
without much luck.  If anyone knows of a good one, can you please point 
me in the right direction?  If I can avoid reinventing some wheels, that 
would be great.


In particular, at the moment I'm looking for a post-commit hook that 
will email *only* the user who made the previous commit.


Thanks!
Peter Ruprecht

Re: How to use GNOME keyring with Subversion

[Mark Phippard]

On Fri, Jan 15, 2010 at 11:33 AM, Giulio Troccoli
 wrote:
> Although I have been using Subversion since 1.2, so for quite a while, I'm 
> very new to GNOME keyring. I would like to set it up but I don't seem able to 
> do that.
>
> I have, for testing, created a VM with CentOS 5.4. Then I have built 
> Subversion 1.6.6 from source with the --with-gnome-keyring option.
>
> The gnome-keyring-daemon is running (does it need any parameters?) and the 
> keyring manager show an unlocked session called "session".
>
> I have change the config file to use gnome-keyring and the servers files with 
> store-passwords = yes and store-plaintext-passwords = no.
>
> Still, when I update an existing wc I am asked for the Subversion password 
> every time.
>
> As I said I am totally new to keyring and stuff like that, so if anyone has 
> successfully set it up please help me :-)
>
> Of course, if you need more information just ask

The CollabNet RPM has gnome-keyring support and installs into a
private location (/opt/CollabNet_Subversion) so it will not mess up
your other build or install.  I'd suggest you try it to rule out
problems with your build.

I assumed you made sure you were using your build and not the version
that comes with CentOS.


-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

RE: malformed file problem -- version 1.5.7

[James D. Parra]

 
> Is there a way that I can repair this?
No idea, sorry.  But:

> Is there a way to roll back to revision 11529 and start there? 

Try svnadmin dump with the -r parameter to dump just the revisions you
want to keep, then svnadmin load to load them into a new repository.
Any checkouts of r11529 or earlier should be OK; if you have a later
checkout then you'll need to delete it and re-checkout.

> Should I remove all the revision number files greater that 11529

Hand-editing the repository sounds like a bad idea*... you might get rid
of this corruption but introduce a different corruption.
Using dump/load should give you a valid repository.

Kind regards,

Jon

[*] It's number zero on the "Subversion Worst Practices" guide:
http://www.red-bean.com/fitz/presentations/2007-07-27-OSCON-svn-worst-pr
actices.pdf
~~

Thank you, Jon, for the info. 

Before I do such a major rollback, is there a way to rollback a particular
project within the repos without affecting other projects or, to be more
concise, maintain revision numbers for the other projects that are not
affected by the mal-formed file error?

Thanks again for your help.

James 

Re: Post-commit hook recipes?

[Rob van Oostrum]

http://svn.apache.org/viewvc/subversion/trunk/tools/hook-scripts/

On Fri, Jan 15, 2010 at 11:34 AM, Peter Ruprecht <
rupr...@jilau1.colorado.edu> wrote:

> Hi,
>
> I've been looking for a collection of example post-commit hook scripts
> without much luck.  If anyone knows of a good one, can you please point me
> in the right direction?  If I can avoid reinventing some wheels, that would
> be great.
>
> In particular, at the moment I'm looking for a post-commit hook that will
> email *only* the user who made the previous commit.
>
> Thanks!
> Peter Ruprecht
>

Errors while checking out large directory

[Andy Levy]

I've found several references to this problem via Google & the list
archives, but never any answers.

When checking out from *some* clients (one in particular), I get the
following errors in my Apache error log on a regular basis:

[Fri Jan 15 12:33:57 2010] [error] [client IP] Provider encountered an
error while streaming a REPORT response.  [500, #0]
[Fri Jan 15 12:33:57 2010] [error] [client IP] A failure occurred
while driving the update report editor  [500, #190004]

My SVN server is CollabNet's distribution of 1.5.2 running on Windows
2003 with the default configuration of Apache that's included.

We have the most difficulty when performing a build using AnthillOS on
another Windows 2003 server. HOWEVER, I have three servers running
identical versions of AnthillOS, checking out from the same SVN
server. Only one of them generates these errors.

I have multiple copies of my project being built on each server, but
they are done sequentially. Sometimes one or two of the builds will
work OK; eventually it'll get to the point where all of them fail. I'm
left with a partial checkout of the WC; if I run svn update manually
while logged into the server, it completes successfully.

If I restart Apache, it seems to run OK...for a while. I have a
checkout running right now and I'm showing httpd using about 33MB of
memory.

This is slowing our development processes significantly, as we're
forced to re-attempt each build multiple times. Our change management
processes are designed around doing the builds (which also involve
tagging) through AnthillOS, so I can't just push it through manually
without additional administrative overhead.

Has anyone found any resolution to this? I've seen references to this
happening as recently as SVN 1.6.1. Do I have to schedule regular
restarts of the service? Is there a configuration option I can change?

Re: Errors while checking out large directory

[Andy Levy]

On Fri, Jan 15, 2010 at 15:08, Andy Levy  wrote:
> I've found several references to this problem via Google & the list
> archives, but never any answers.
>
> When checking out from *some* clients (one in particular), I get the
> following errors in my Apache error log on a regular basis:
>
> [Fri Jan 15 12:33:57 2010] [error] [client IP] Provider encountered an
> error while streaming a REPORT response.  [500, #0]
> [Fri Jan 15 12:33:57 2010] [error] [client IP] A failure occurred
> while driving the update report editor  [500, #190004]
>
> My SVN server is CollabNet's distribution of 1.5.2 running on Windows
> 2003 with the default configuration of Apache that's included.
>
> We have the most difficulty when performing a build using AnthillOS on
> another Windows 2003 server. HOWEVER, I have three servers running
> identical versions of AnthillOS, checking out from the same SVN
> server. Only one of them generates these errors.
>
> I have multiple copies of my project being built on each server, but
> they are done sequentially. Sometimes one or two of the builds will
> work OK; eventually it'll get to the point where all of them fail. I'm
> left with a partial checkout of the WC; if I run svn update manually
> while logged into the server, it completes successfully.
>
> If I restart Apache, it seems to run OK...for a while. I have a
> checkout running right now and I'm showing httpd using about 33MB of
> memory.
>
> This is slowing our development processes significantly, as we're
> forced to re-attempt each build multiple times. Our change management
> processes are designed around doing the builds (which also involve
> tagging) through AnthillOS, so I can't just push it through manually
> without additional administrative overhead.
>
> Has anyone found any resolution to this? I've seen references to this
> happening as recently as SVN 1.6.1. Do I have to schedule regular
> restarts of the service? Is there a configuration option I can change?
>

Sorry, replying to myself as I omitted what may be important information.

The checkout always fails while working on a directory holding (at the
moment) 956 items (PDF files), totaling 663MB.

Re: Errors while checking out large directory

[Mark Phippard]

On Fri, Jan 15, 2010 at 3:08 PM, Andy Levy  wrote:
> I've found several references to this problem via Google & the list
> archives, but never any answers.
>
> When checking out from *some* clients (one in particular), I get the
> following errors in my Apache error log on a regular basis:
>
> [Fri Jan 15 12:33:57 2010] [error] [client IP] Provider encountered an
> error while streaming a REPORT response.  [500, #0]
> [Fri Jan 15 12:33:57 2010] [error] [client IP] A failure occurred
> while driving the update report editor  [500, #190004]
>
> My SVN server is CollabNet's distribution of 1.5.2 running on Windows
> 2003 with the default configuration of Apache that's included.

This is usually an issue with the client where it is not responding
fast enough and the server thinks the connection has been dropped.
For example, the client can spend so long writing the working copy
that the server thinks it dropped.

http://subversion.tigris.org/faq.html#secure-connection-truncated

There is a timeout in the httpd.conf you can set to make the server
wait longer.  I do not recall what it is off the top of my head.

I think if you changed the client to use Serf instead of Neon you
would also avoid this problem.  Neon does this all as one giant HTTP
request, where as Serf breaks it up into lots of smaller GET requests.
 So I do not think Serf would likely timeout.

You should be able to just edit the %AppData%\Subversion\servers file
on the client to tell it to use Serf over Neon (assuming your binaries
support it).

[global]
http-library = serf

If Anthill uses SVNKit then I am not sure what you can do other than
increase the timeout on the server.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

How to make an encrypted svnsynced repository

[Ton Boelens]

Hi,

At the moment, my SVN repository is snvsynced to a server in another
location. I would like to make this remote repository encrypted on the
file level, so that even somebody who has physically access to this
server, cannot read the contents of the files.

I have searched in the svn manual, with Google and in the past couple
of months posts I have of this mailing list, but I could find no
reference.

Does that mean that there is no way to design a solution to this
requirement?

 
---
Ton Boelens
's-Hertogenbosch, The Netherlands
TonsofTime, enjoy the Now http://tonsoftime.com

Re: How to make an encrypted svnsynced repository

[Andy Levy]

On Fri, Jan 15, 2010 at 16:23, Ton Boelens  wrote:
> Hi,
>
> At the moment, my SVN repository is snvsynced to a server in another
> location. I would like to make this remote repository encrypted on the
> file level, so that even somebody who has physically access to this
> server, cannot read the contents of the files.
>
> I have searched in the svn manual, with Google and in the past couple
> of months posts I have of this mailing list, but I could find no
> reference.
>
> Does that mean that there is no way to design a solution to this
> requirement?

Wouldn't disk/file-level encryption be a task for the OS? IOW,
encrypt/decrypt the data as it's written to/read from disk, regardless
of whether it's a repository or not?

Re: How to make an encrypted svnsynced repository

[Alec Kloss]

On 2010-01-15 22:23, Ton Boelens wrote:
> Hi,
> 
> At the moment, my SVN repository is snvsynced to a server in another
> location. I would like to make this remote repository encrypted on the
> file level, so that even somebody who has physically access to this
> server, cannot read the contents of the files.
> 
> I have searched in the svn manual, with Google and in the past couple
> of months posts I have of this mailing list, but I could find no
> reference.
> 
> Does that mean that there is no way to design a solution to this
> requirement?

I don't think this is built into subversion.  I've asked about a
similar feature in the past and not gotten anywhere.  It would be
pretty slick to have a "repository session key" that is
pgp-encrypted for the committers/reviewers of the repository that
all files (and network traffic) is encrypted with.  If the svn
clients managed it all well, it could be pretty seamless.  A new
committer would be added to the repository session key, and
revoking a committer would require generating a new key and
encrypting new revisions with it.   It would be a great feature
because you could distribute a secure repository onto a public
subversion server and only send private data to and from it.

-- 
alec.kl...@oracle.com   Oracle Middleware
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x432B9956


pgpwtHRj5tcGa.pgp
Description: PGP signature

RE: sync bug -> corrupted proxy repo

[Andersen, Krista]

Thank you Jon, for your explanation and workaround.

Are there any "best practices" that we can advise our dev groups to follow to 
avoid this problem?

Otherwise, your suggestions seem to indicate I would have to run the sync on a 
cronjob and not with the hook script.  That is something we would like to 
avoid.  So I have added a start time comparison and sleep in a start-commit 
hook instead.  Do you see any reason why this would cause other problems?

I am assuming that if the commits must start at least one second apart - then 
the sync from the post-commit hook would not be able to reach a race condition. 
 Is this a reasonable assumption?

#!/usr/bin/sh

# START-COMMIT HOOK
# kanderse Jan 13, 2010

# The start-commit hook is invoked before a Subversion txn is created
# in the process of doing a commit.

# This script checks the start time and compares with the start time
# of the previous commit.  It will cause a commit to wait one second if
# the last commit was started less than one second earlier.

# The purpose of this wait is to prevent known issue 3546 [1][2].
# a race condition involving multiple sync processes running at
# the same time that result in a corrupted proxy.

REPOS="$1"
USER="$2"

DATE1=`cat /$REPOS/hooks/start-time.txt` # previous start time
DATE2=`/usr/local/bin/date +%s` # record current start time
echo $DATE2 > /$REPOS/hooks/start-time.txt
# echo $DATE2 $DATE1 `expr $DATE2 - $DATE1`

if [ `expr $DATE2 - $DATE1` -lt 1 ]
then sleep 1 # to prevent sync race that results in sync duplication and 
corrupted proxy
fi
# All checks passed, so allow the commit.
exit 0

Krista Andersen

Global Development Infrastructure
Investment Technology Group, Inc.
400 Corporate Pointe, 8th Floor
Culver City, CA 90230
Direct: 213.270.7570



-Original Message-
From: Jon Foster [mailto:jon.fos...@cabot.co.uk]
Sent: Wednesday, January 13, 2010 5:13 AM
To: Andersen, Krista; users@subversion.apache.org
Cc: ssi-svn_admin
Subject: RE: sync bug -> corrupted proxy repo

Hi,

Andersen, Krista [mailto:krista.ander...@itg.com] wrote:
> Twice I have seen one of my proxy repositories become corrupted due
> to an apparent bug in the svnsync sync process.  Has anyone else
> seen this type of behavior from Subversion?

This is probably caused by issue 3546 [1][2].  This is a race
condition - if you have several sync processes running at the same
time then the mirror can get corrupted.  You had three commits which
were 1 second apart, so your hook script started 3 copies of svnsync
within 2 seconds.  I think this is the first practical report of this
bug; previous discussion was theoretical.

> Here is a comparison the output of the svn log -v for the offending
> revisions (324,325) on both the corrupted and non-corrupted proxy
> repo.

It looks like rev 323 got mirrored twice (as mirror revs 323 and 324),
then rev 324 was mirrored (as mirror rev 325).

> I am a bit concerned about the stability of Subversion since this
> is the second time in two months that I have had to fix this issue.
> Is there a patch or something to prevent this in the future?

Suggested workaround: Change your hook scripts to use the lockf or
lockfile tools[3] to ensure that only one instance of svnsync runs
at once.

Kind regards,

Jon

[1]
http://mail-archives.apache.org/mod_mbox/subversion-dev/200911.mbox/%3C2
0091127115356.gc9...@jack.stsp.name%3e

[2] http://subversion.tigris.org/issues/show_bug.cgi?id=3546

[3]
http://mail-archives.apache.org/mod_mbox/subversion-dev/200911.mbox/%3C2
0091127132659.ge9...@jack.stsp.name%3e



**
This email and its attachments may be confidential and are intended solely for 
the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those 
of Cabot Communications Ltd.

If you are not the intended recipient of this email and its attachments, you 
must take no action based upon them, nor must you copy or show them to anyone.

Cabot Communications Limited
Verona House, Filwood Road, Bristol BS16 3RY, UK
+44 (0) 1179584232

Co. Registered in England number 02817269

Please contact the sender if you believe you have received this email in error.

**


__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email
__
-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
This message is for the named person's use only. This communication is for
informational purposes only and has been obtained from sources believed to
be reliable, but it is not necessarily complete and its accuracy cannot be
guaranteed. It is not intended as an offer or solicitation fo

Re: Keyword expansion on Unicode text files

[Rob van Oostrum]

2010/1/15 Ulrich Eckhardt 

> On Thursday 14 January 2010, Ryan Schmidt wrote:
> > On Jan 14, 2010, at 13:15, Harald-René Flasch wrote:
> > > what should I do in order to make keyword expansion on Unicode text
> files
> > > working (e.g. .inf files)?
> > >
> > > - BOM is FF FE (used Notepad Save As --> Unicode to create the file)
> > > - There is only the svn:keywords property on the file
> >
> > Sorry, AFAIK there still isn't a way to make svn:keywords work in UTF-16
> > files.
>
> That said, you might be able to store them as UTF-8, which is another full
> Unicode-capable format and even more common than the used UTF-16. Using
> UTF-8, keywork expansion simply works.
>
> Uli
>

AFAIK, the root of the issue is SVN treating UTF-16 files as binary. Once
that's resolved, keyword expansion will work automagically. I have no idea
how complicated this would be to resolve, but my guess is that it's
non-trivial or it would have never been an issue in the first place.

R.

Re: Keyword expansion on Unicode text files

[Ryan Schmidt]

On Jan 15, 2010, at 03:07, Harald-René Flasch wrote:

> It shouldn't be a unresolvable programming issue to support keyword expansion 
> for Unicode text files?

I agree, it shouldn't be unresolvable. But at present it is unresolved.

Re: How to make an encrypted svnsynced repository

[Ton Boelens]

On Fri, Jan 15, 2010 at 03:37:19PM -0600, Alec Kloss wrote:
> On 2010-01-15 22:23, Ton Boelens wrote:
> > Hi,

> > At the moment, my SVN repository is snvsynced to a server in another
> > location. I would like to make this remote repository encrypted on the
> > file level, so that even somebody who has physically access to this
> > server, cannot read the contents of the files.

> > I have searched in the svn manual, with Google and in the past couple
> > of months posts I have of this mailing list, but I could find no
> > reference.

> > Does that mean that there is no way to design a solution to this
> > requirement?

> I don't think this is built into subversion.  I've asked about a
> similar feature in the past and not gotten anywhere.  It would be
> pretty slick to have a "repository session key" that is
> pgp-encrypted for the committers/reviewers of the repository that
> all files (and network traffic) is encrypted with.  If the svn
> clients managed it all well, it could be pretty seamless.  A new
> committer would be added to the repository session key, and
> revoking a committer would require generating a new key and
> encrypting new revisions with it.   It would be a great feature
> because you could distribute a secure repository onto a public
> subversion server and only send private data to and from it.

I agree, that would be great functionality. However, I would like to
have the encrypted remote copy this winter :-)
 

Ton

Re: How to make an encrypted svnsynced repository

[Ton Boelens]

On Fri, Jan 15, 2010 at 04:35:08PM -0500, Andy Levy wrote:
> On Fri, Jan 15, 2010 at 16:23, Ton Boelens  wrote:
> > Hi,

> > At the moment, my SVN repository is snvsynced to a server in
> > another location. I would like to make this remote repository
> > encrypted on the file level, so that even somebody who has
> > physically access to this server, cannot read the contents of the
> > files.

> > I have searched in the svn manual, with Google and in the past
> > couple of months posts I have of this mailing list, but I could
> > find no reference.

> > Does that mean that there is no way to design a solution to this
> > requirement?

> Wouldn't disk/file-level encryption be a task for the OS? IOW,
> encrypt/decrypt the data as it's written to/read from disk,
> regardless of whether it's a repository or not?

Andy,
If you have physical access to a server that contains the encrypted
disk and the encryption key as well, you can still use the key to open
the disk. How could I solve that?

What we are thinking about is something like this (it's a snippit of a
larger script):

/usr/bin/openssl enc -d -aes-256-cbc -in $1 -out "$BESTAND" -pass file:$PAD/$KEY

This command will create a local copy of the repository, which will
then be rsynced to the remote server*.

I wonder if when I execute this command to the files of a SVN
repository and direct the output to another directory, if the new
directory, when restored (decrypted again) would still be a good
useable repo. I guess we have to do some testing.

Has anybody tried something like this?

 *(this sync would function only as some kind of last resort backup) 

Thank you for your reply!


---
Ton Boelens
's-Hertogenbosch, The Netherlands