Re: Expected performance
Guten Tag Andreas Krey, am Dienstag, 9. Juli 2013 um 07:31 schrieben Sie: > 9550 Files, half a GB wc size, 15 seconds. > You may want to use another file system? Or your hardware and connection to your repo with it's server etc. I vote for the latter and claim that you didn't mention little details like an SSD for the working copy etc. Of course only because those would have only little impact on checkout performance... ;-) Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Parallel checkout within checkout
On Mon, Jul 08, 2013 at 10:51:54PM -0500, Frank Loeffler wrote: > Would this now be a good time to open a ticket to try to get this in a > future release version? No need, I'll try to take care of this fix myself. Thanks for your report!
Re: Expected performance
On Tue, 09 Jul 2013 09:52:22 +, Thorsten Schöning wrote: ... > am Dienstag, 9. Juli 2013 um 07:31 schrieben Sie: > > > 9550 Files, half a GB wc size, 15 seconds. > > > You may want to use another file system? > > Or your hardware and connection to your repo with it's server etc. I > vote for the latter and claim that you didn't mention little details > like an SSD for the working copy etc. Of course only because those > would have only little impact on checkout performance... ;-) No, that is an admittedly beefy machine with rotating rust (although possibly raided), and GBit network access to the server. Neither does the server have SSDs. But the 1.5MByte/s that gets mentioned here seems quite unambitious; with halfways decent hardware you should be able, in an svn checkout, to saturate any network below 100MBit/s - the machine now under my desk writes a tree of 10 files and 7GB in about a minute (that is not an svn checkout). Andreas -- "Totally trivial. Famous last words." From: Linus Torvalds Date: Fri, 22 Jan 2010 07:29:21 -0800
Re: Expected performance
Guten Tag Andreas Krey, am Dienstag, 9. Juli 2013 um 11:02 schrieben Sie: > the machine now under my desk > writes a tree of 10 files and 7GB in about a minute (that is > not an svn checkout). And that's the hardware I wanted to read about, there surely is some advanced RAID or SSD used and no, this would not (yet) be common hardware. My Crucial M4 SSD for example is pushing the bottleneck to my 6 year old CPU, instead of NTFS itself. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Re: Expected performance
On Tue, Jul 9, 2013 at 1:31 AM, Andreas Krey wrote: > On Mon, 08 Jul 2013 14:33:03 +, Andy Levy wrote: > >> I just checked out 2400 files, about 1.7GB, and it took just over 19 minutes. >> >> Client I/O speed is a big factor (7200RPM hard drive w/ NTFS in my case). > > 9550 Files, half a GB wc size, 15 seconds. > > You may want to use another file system? Got any good recommendations for Windows 7 that will also be compatible with my Windows Server 2008 R2 systems *and* my sysadmins will accept?
svnsync crashes on a huge commit
svnsync failed to sync repository with a single commit containing 56000 files. This commit handled perfectly by console svn and tortoise svn. We can checkout this folder structure and so on. But when I tried to sync this commit to the local mirror repository it fails. 32 bit version of svnsync crashed with out of memory exception. 64 bit version made a transaction folder containing 117000 files in it, allocated like 3gb of memory and finally crashed with this diagnostic: "svn: E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: internal malfunction". May be this commit is not an example of the best practice but it crash only svnsync all the other svn tools handle it perfectly. What should I do? How to mirror this repository? Is it a bug or not?
Check-out fails with LANG=C
When checking out one of the subsystems from our repository, the check-out always fails for one of my colleagues with the following error messages after about 2/3 of the files: svn: E155009: Failed to run the WC DB work queue associated with '$HOME/C8-32-C-serf/CCS/db/src', work item 9694 (file-install CCS/db/src/dbBackupSingleAttr.c 1 0 1 1) svn: E12: Can't create a character converter from 'UTF-8' to native encoding The same check-out works for me without any problems using the same svn client on the same machine. The difference is in the setting of the LANG environment variable. When set to "en_US.UTF-8", everything works as it should, but with LANG=C, the check-out always fails. The svn client is version 1.8.0 (from the CollabNet rpms), 32-bit(!), running on a 64-bit Linux installation. I ran several experiments with both 32-bit and 64-bit installs of 1.7.10 and 1.8.0, all on the same and all checking out to a local disk, with both language settings. The results are below. Summary: the 64-bit clients worked in both versions, only the 32-bit 1.8.0 client fails. The 1.7.10 client failed with a different error when I tried to use serf for better comparison to the 1.8.0 client, but worked fine with the default neon library. We would like to switch to 1.8.x but are unfortunately forced to run 32-bit clients because we need to interface with a 32-bit Exclipse installation for vxWorks. But I don't see why 64-bit client should be required in the first place. A simple thing like a check-out should not consume large amounts of memory. I suspect that there is some kind of memory leak here. The results below show that 1.8.0 requires much more memory than 1.7.10 with neon (see the numbers for "maxresident"). - Michael The language settings used in the commands below: l1=en_US.UTF-8 l2=C The check-out is always from the same URL to a fresh working copy. The name of the wc is on the line below the command. The following lines are the output of the "time" command or the error message from svn. 1.7.10 x64 -- LANG=$l1 time svn co $url -r244060 C7-$l1-neon C7-en_US.UTF-8-neon 23.79user 17.07system 2:13.26elapsed 30%CPU (0avgtext+0avgdata 78416maxresident)k 568inputs+1697520outputs (0major+5039minor)pagefaults 0swaps LANG=$l2 time svn co $url -r244060 C7-$l2-neon C7-C17.73user 12.47system 0:35.69elapsed 84%CPU (0avgtext+0avgdata 79008maxresident)k 5208inputs+1696848outputs (0major+5070minor)pagefaults 0swaps -neon LANG=$l1 time svn co $url -r244060 C7-$l21serf --config-option servers:global:http-library=serf svn: E120190: Error running context: APR does not understand this error code 0.01user 0.00system 0:00.03elapsed 64%CPU (0avgtext+0avgdata 16208maxresident)k 0inputs+0outputs (0major+1137minor)pagefaults 0swaps 1.7.10 i386 --- LANG=$l1 time svn co $url -r244060 C7-32-$l1-neon C7-32-en_US.UTF-8-neon 19.79user 11.30system 0:37.14elapsed 83%CPU (0avgtext+0avgdata 71760maxresident)k 1008inputs+1697720outputs (2major+4589minor)pagefaults 0swaps LANG=$l2 time svn co $url -r244060 C7-32-$l2-neon C7-32-C-neon 19.65user 12.69system 0:36.54elapsed 88%CPU (0avgtext+0avgdata 71200maxresident)k 2624inputs+1696488outputs (17major+4547minor)pagefaults 0swaps LANG=$l1 time svn co $url -r244060 C7-32-$l1-serf --config-option servers:global:http-library=serf svn: E120190: Error running context: APR does not understand this error code 0.00user 0.00system 0:00.13elapsed 13%CPU (0avgtext+0avgdata 12496maxresident)k 664inputs+0outputs (5major+881minor)pagefaults 0swaps 1.8.0 x64 - LANG=$l1 time svn co $url -r244060 C8-64-$l1-serf C8-64-en_US.UTF-8-serf 17.40user 12.76system 0:35.28elapsed 85%CPU (0avgtext+0avgdata 391952maxresident)k 1360inputs+1772808outputs (1major+24635minor)pagefaults 0swaps LANG=$l2 time svn co $url -r244060 C8-64-$l2-serf C8-64-C-serf 18.30user 13.53system 0:36.52elapsed 87%CPU (0avgtext+0avgdata 4858336maxresident)k 11992inputs+1771488outputs (50major+303908minor)pagefaults 0swaps 1.8.0 x32 - LANG=$l1 time svn co $url -r244060 C8-32-$l1-serf C8-32-en_US.UTF-8-serf 16.56user 10.61system 0:31.15elapsed 87%CPU (0avgtext+0avgdata 336768maxresident)k 2136inputs+1772992outputs (3major+21161minor)pagefaults 0swaps LANG=$l2 time svn co $url -r244060 C8-32-$l2-serf svn: E155009: Failed to run the WC DB work queue associated with '$HOME/C8-32-C-serf/CCS/db/src', work item 9694 (file-install CCS/db/src/dbBackupSingleAttr.c 1 0 1 1) svn: E12: Can't create a character converter from 'UTF-8' to native encoding 12.61user 9.24system 0:24.90elapsed 87%CPU (0avgtext+0avgdata 3369152maxresident)k 928inputs+1279336outputs (1major+210678minor)pagefaults 0swaps
Re: Check-out fails with LANG=C
On Tue, 09 Jul 2013 12:55:29 +, Michael Pruemm wrote: ... > The difference is in the setting of the LANG environment variable. > When set to "en_US.UTF-8", everything works as it should, but with > LANG=C, the check-out always fails. svn wants to convert file names in the repository to filename on the local disk by using the locally set LANG. This is a rather stupid idea, but that is the way it is. If you have any non-ascii chars in the file names in the repo, no svn client should check out that repo while under LANG=C. Andreas -- "Totally trivial. Famous last words." From: Linus Torvalds Date: Fri, 22 Jan 2010 07:29:21 -0800
Re: Check-out fails with LANG=C
On Tue, Jul 9, 2013 at 3:24 PM, Andreas Krey wrote: > svn wants to convert file names in the repository to filename > on the local disk by using the locally set LANG. This is a > rather stupid idea, but that is the way it is. > Yes, I know. The whole point is that with 1.7.10 it works, but with 1.8.0 it doesn't. It seems to me that svn tries to generate lots of those "tranlators" instead of creating one and reusing it. > If you have any non-ascii chars in the file names in the > repo, no svn client should check out that repo while > under LANG=C. > > No, we don't have those. All file names (at least in that part of the repository) are plain ASCII. - Michael
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 03:24:15PM +0200, Andreas Krey wrote: > On Tue, 09 Jul 2013 12:55:29 +, Michael Pruemm wrote: > ... > > The difference is in the setting of the LANG environment variable. > > When set to "en_US.UTF-8", everything works as it should, but with > > LANG=C, the check-out always fails. > > svn wants to convert file names in the repository to filename > on the local disk by using the locally set LANG. This is a > rather stupid idea, but that is the way it is. > > If you have any non-ascii chars in the file names in the > repo, no svn client should check out that repo while > under LANG=C. > It looks like a memory leak. The client fails to allocate memory when setting up charset translation. I would say the LC_CTYPE issue is a red herring. There is some other problem that makes the client run out of memory. Michael, does this only happen with ra_serf? Can you please test with svn://, or svn+ssh://, or file:// access?
Re: Parallel checkout within checkout
On Tue, Jul 09, 2013 at 10:03:30AM +0200, Stefan Sperling wrote: > On Mon, Jul 08, 2013 at 10:51:54PM -0500, Frank Loeffler wrote: > > Would this now be a good time to open a ticket to try to get this in a > > future release version? > > No need, I'll try to take care of this fix myself. > Thanks for your report! There are some open questions about this fix. I've filed http://subversion.tigris.org/issues/show_bug.cgi?id=4390 Please add yourself to the Cc list there if you want to stay informed about progress on this issue.
Re: Check-out fails with LANG=C
On Tue, Jul 9, 2013 at 3:44 PM, Stefan Sperling wrote: > Michael, does this only happen with ra_serf? Can you please test > with svn://, or svn+ssh://, or file:// access? > Here are the results with 32-bit 1.8.0 using svn+ssh: LANG=$l1 time svn co svn+ssh://... -r244060 C8-32-$l1-svnssh 17.71user 11.13system 1:05.59elapsed 43%CPU (0avgtext+0avgdata 325792maxresident)k 4920inputs+1764088outputs (29major+51815minor)pagefaults 0swaps LANG=$l2 time svn co svn+ssh://... -r244060 C8-32-$l2-svnssh (core dump) 11.71user 10.44system 1:13.20elapsed 30%CPU (0avgtext+0avgdata 3554080maxresident)k 1784inputs+3064200outputs (13major+223537minor)pagefaults 0swaps Note: instead of the error message I got a core dump. That happened before, too. The check-out went about as far as in the other failing cases. So the problem appears to be somewhat independent of the access method. To test file:// access, I need to get the repository to a different machine first. This may take a bit... - Michael
Re: Check-out fails with LANG=C
On 09.07.2013 15:24, Andreas Krey wrote: > On Tue, 09 Jul 2013 12:55:29 +, Michael Pruemm wrote: > ... >> The difference is in the setting of the LANG environment variable. >> When set to "en_US.UTF-8", everything works as it should, but with >> LANG=C, the check-out always fails. > svn wants to convert file names in the repository to filename > on the local disk by using the locally set LANG. This is a > rather stupid idea, but that is the way it is. Since we're on the topic, would you care to explain why translating files to the native encoding is "a rather stupid idea"? Would it be better to leave files in an encoding that the rest of the OS doesn't understand? -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
For file:// access I had to mount the repository via NFS, and access is much slower. LANG=$l1 time svn co file:///... -r244060 C8-32-$l1-file C8-32-en_US.UTF-8-file 32.32user 21.51system 5:36.36elapsed 16%CPU (0avgtext+0avgdata 519408maxresident)k 1101632inputs+1799224outputs (4major+36982minor)pagefaults 0swaps LANG=$l2 time svn co file:///...-r244060 C8-32-$l2-file C8-32-C-file (core dump) 18.69user 14.67system 2:04.79elapsed 26%CPU (0avgtext+0avgdata 3398640maxresident)k 125864inputs+2651200outputs (29major+213703minor)pagefaults 0swaps It fails with a core dump again, but earlier than with the other access methods. A full check-out results in a wc of ~450MB. Here are the file counts for the various check-out attempts with svn 1.8.0: $ for d in C8*; do echo "$(find $d | wc -l) $d"; done 16658 C8-32-C-file 22359 C8-32-C-serf 22372 C8-32-C-svnssh 30526 C8-32-en_US.UTF-8-file 30526 C8-32-en_US.UTF-8-serf 30526 C8-32-en_US.UTF-8-svnssh 30526 C8-64-C-serf 30526 C8-64-en_US.UTF-8-serf - Michael
Re: Check-out fails with LANG=C
On Tue, 09 Jul 2013 16:26:40 +, Branko ??ibej wrote: ... > Since we're on the topic, would you care to explain why translating > files to the native encoding is "a rather stupid idea"? Because LANG isn't "the native encoding", but, for the file system the "naive encoding". What encoding is used in the file system is a property of the file system or even individual directories and files, but certainly not of the terminal session I'm using to do the checkout. (And if you think that LANG should apply to file names on disk, why would you think it should not do so for the file names that come from the svn server in the checkout?) Andreas -- "Totally trivial. Famous last words." From: Linus Torvalds Date: Fri, 22 Jan 2010 07:29:21 -0800
"svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
"svn add" is having trouble with *.png files. This is with a 1.8 and a 1.7.9 client. I create a new test repository, copy in some vender code, then when I run "svn add" I get the following error on 1.8: svn add build-pipeline-plugin-1.3.3 ... A build-pipeline-plugin-1.3.3\src\main\resources\index.jelly A build-pipeline-plugin-1.3.3\src\main\webapp A build-pipeline-plugin-1.3.3\src\main\webapp\images svn: E29: Can't set 'svn:eol-style': file 'C:\temp\1.8\test18\tags\build-pipeline-plugin-1.3.3\src\main\webapp\images\gear.png' has binary mime type property When I revert and re-run the add, the add will sometimes fail on a different *.png file, which is odd. Under 1.7.9, I get one of two errors, either: svn: E29: File 'C:\temp\test18\build-pipeline-plugin-1.3.3\src\main\webapp\images\application-small-list-blue.png' has binary mime type property or svn: E29: File 'C:\temp\test18\build-pipeline-plugin-1.3.3\src\main\webapp\css\redmond\images\ui-icons_f9bd01_256x240.png' has inconsistent newlines svn: E135000: Inconsistent line ending style This is on Windows 7 32-bit with "svn, version 1.7.9 (r1462340)" or "svn, version 1.8.0 (r1490375)" clients from Collabnet. 'svn import' imports the files without issue. Enabling auto-props and uncommenting the '*.png' auto-prop in the client's config file made no difference either. Anyone have any ideas? Steps to reproduce: Create a new repo: 1. cd c:\repos 2. svnadmin create test18 3. svnserve -d -r c:\repos 4. edit test18\conf\svnserve.conf set "anon-access = write" in the [General] section Create a new workspace: 5. cd c:\temp\ 6. svn co svn://localhost/test18 7. cd test18 Get the files: 8. unzip build-pipeline-plugin-1.3.3.zip Zip is available here: https://docs.google.com/file/d/0B-xhQuXzTnK8VnJmNS1KX2NmZDg/edit?usp=sharing Or you can install a Mercurial (Hg) client and hg clone https://code.google.com/p/build-pipeline-plugin/ build-pipeline-plugin-1.3.3 cd build-pipeline-plugin-1.3.3 hg co build-pipeline-plugin-1.3.3 rd /s/q .hg del .hgignore .hgtags cd .. xcopy /s/e/v/i build-pipeline-plugin-1.3.3 test18/build-pipeline-plugin-1.3.3 Run the add: 9. svn add build-pipeline-plugin-1.3.3 10. Extra credit, revert and run again to see it fail on different files: svn revert -R build-pipeline-plugin-1.3.3 & svn add build-pipeline-plugin-1.3.3
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 04:34:02PM +0200, Michael Pruemm wrote: > For file:// access I had to mount the repository via NFS, and access is > much slower. > > > LANG=$l1 time svn co file:///... -r244060 C8-32-$l1-file > C8-32-en_US.UTF-8-file > 32.32user 21.51system 5:36.36elapsed 16%CPU (0avgtext+0avgdata > 519408maxresident)k > 1101632inputs+1799224outputs (4major+36982minor)pagefaults 0swaps > > LANG=$l2 time svn co file:///...-r244060 C8-32-$l2-file > C8-32-C-file > (core dump) > 18.69user 14.67system 2:04.79elapsed 26%CPU (0avgtext+0avgdata > 3398640maxresident)k > 125864inputs+2651200outputs (29major+213703minor)pagefaults 0swaps > > It fails with a core dump again, but earlier than with the other access > methods. > > A full check-out results in a wc of ~450MB. Here are the file counts for > the various check-out attempts with svn 1.8.0: > > $ for d in C8*; do echo "$(find $d | wc -l) $d"; done > 16658 C8-32-C-file > 22359 C8-32-C-serf > 22372 C8-32-C-svnssh > 30526 C8-32-en_US.UTF-8-file > 30526 C8-32-en_US.UTF-8-serf > 30526 C8-32-en_US.UTF-8-svnssh > 30526 C8-64-C-serf > 30526 C8-64-en_US.UTF-8-serf > > - Michael Michael, Can you please file a new issue pointing to this thread? See http://subversion.apache.org/reporting-issues.html Thanks!
Re: "svn add *" and sign in filename
Run 'svn add wc/b...@2.txt@'. That's the documented workaround for escaping @, since it's used for the peg revision syntax. Could 'svn add' not parse '@' in filenames specially? Well, yes. But apparently, it doesn't behave that way in 1.8.0, and the above is the workaround. Варфоломеев Игорь wrote on Tue, Jul 09, 2013 at 20:12:00 +0400: > Hi all, > > I think I ran into another issue, related to @-signs in filename. > In particular > svn add wc\B\* > results in > svn: E29: 'wc/b...@2.txt': a peg revision is not allowed here > if there's a file "@2.txt" inside "B". > > Here's a script to reproduce this: > https://skydrive.live.com/redir?resid=8FD4EC3161ABA67A!941 > (the same file is also attached to the message) > > Can anyone confirm the issue? > > Best regards, > Igor Varfolomeev >
Re: svnsync crashes on a huge commit
On Tue, Jul 09, 2013 at 03:30:44PM +0400, Anatoly Zapadinsky wrote: > svnsync failed to sync repository with a single commit containing 56000 > files. This commit handled perfectly by console svn and tortoise svn. We > can checkout this folder structure and so on. But when I tried to sync this > commit to the local mirror repository it fails. > 32 bit version of svnsync crashed with out of memory exception. > 64 bit version made a transaction folder containing 117000 files in it, > allocated like 3gb of memory and finally crashed with this diagnostic: > "svn: E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: > internal malfunction". > May be this commit is not an example of the best practice but it crash only > svnsync all the other svn tools handle it perfectly. > What should I do? How to mirror this repository? Is it a bug or not? This problem sounds like a memory leak. I wonder if you're running into the same problem this user is seeing: http://svn.haxx.se/users/archive-2013-07/0128.shtml
RE: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
To add insult to injury, TortoiseSVN's add just died on a .css file: C:\temp\test18\foo\src\main\webapp\css C:\temp\test18\foo\src\main\webapp\css\jquery.fancybox-1.3.4.css File 'C:\temp\test18\foo\src\main\webapp\css\jquery.tooltip.css' has inconsistent newlines Inconsistent line ending style Additional errors: Inconsistent line ending style TortoiseSVN 1.8.0, Build 24401 - 32 Bit , 2013/06/17 18:15:59
Re: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
Andrew Reedick wrote on Tue, Jul 09, 2013 at 13:03:10 -0400: > "svn add" is having trouble with *.png files. This is with a 1.8 and a 1.7.9 > client. > > I create a new test repository, copy in some vender code, then when I run > "svn add" I get the following error on 1.8: > svn add build-pipeline-plugin-1.3.3 > ... > A build-pipeline-plugin-1.3.3\src\main\resources\index.jelly > A build-pipeline-plugin-1.3.3\src\main\webapp > A build-pipeline-plugin-1.3.3\src\main\webapp\images > svn: E29: Can't set 'svn:eol-style': file > 'C:\temp\1.8\test18\tags\build-pipeline-plugin-1.3.3\src\main\webapp\images\gear.png' > has binary mime type property > > When I revert and re-run the add, the add will sometimes fail on a different > *.png file, which is odd. > > Under 1.7.9, I get one of two errors, either: > svn: E29: File > 'C:\temp\test18\build-pipeline-plugin-1.3.3\src\main\webapp\images\application-small-list-blue.png' > has binary mime type property > or > svn: E29: File > 'C:\temp\test18\build-pipeline-plugin-1.3.3\src\main\webapp\css\redmond\images\ui-icons_f9bd01_256x240.png' > has inconsistent newlines > svn: E135000: Inconsistent line ending style > > This is on Windows 7 32-bit with "svn, version 1.7.9 (r1462340)" or "svn, > version 1.8.0 (r1490375)" clients from Collabnet. > > 'svn import' imports the files without issue. Enabling auto-props and > uncommenting the '*.png' auto-prop in the client's config file made no > difference either. > > Anyone have any ideas? Can you try setting "*.png =" in the auto-props section? Perhaps your registry has a setting that that will override. That said, perhaps your registry has a setting for "*.pn*" which that will _not_ override; so check your registry: "REGISTRY:HKLM\\Software\\Tigris.org\\Subversion\\Config" "REGISTRY:HKCU\\Software\\Tigris.org\\Subversion\\Config" For the archives, if this hadn't been a new repository, I'd have suggested to look for inherited svn:auto-props properties too: % svn pg --show-inherited-props -R svn:auto-props
Re: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
Probably caused by auto-props setting an svn:eol-style property. If tortoise actually crashed, please report that to http://tortoisesvn.net/support.html Andrew Reedick wrote on Tue, Jul 09, 2013 at 13:14:12 -0400: > To add insult to injury, TortoiseSVN's add just died on a .css file: > C:\temp\test18\foo\src\main\webapp\css > C:\temp\test18\foo\src\main\webapp\css\jquery.fancybox-1.3.4.css > File 'C:\temp\test18\foo\src\main\webapp\css\jquery.tooltip.css' has >inconsistent newlines > Inconsistent line ending style > Additional errors: > Inconsistent line ending style > > TortoiseSVN 1.8.0, Build 24401 - 32 Bit , 2013/06/17 18:15:59 > >
Re: svnsync crashes on a huge commit
Stefan Sperling wrote on Tue, Jul 09, 2013 at 19:13:32 +0200: > On Tue, Jul 09, 2013 at 03:30:44PM +0400, Anatoly Zapadinsky wrote: > > svnsync failed to sync repository with a single commit containing 56000 > > files. This commit handled perfectly by console svn and tortoise svn. We > > can checkout this folder structure and so on. But when I tried to sync this > > commit to the local mirror repository it fails. > > 32 bit version of svnsync crashed with out of memory exception. > > 64 bit version made a transaction folder containing 117000 files in it, > > allocated like 3gb of memory and finally crashed with this diagnostic: > > "svn: E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: > > internal malfunction". > > May be this commit is not an example of the best practice but it crash only > > svnsync all the other svn tools handle it perfectly. > > What should I do? How to mirror this repository? Is it a bug or not? > > This problem sounds like a memory leak. > I wonder if you're running into the same problem this user is seeing: > http://svn.haxx.se/users/archive-2013-07/0128.shtml A bit of a shot in the dark, but: did you run out of disk space?
Re: svn move and sign in filepath
I agree that it's a bug that, given a file 'foo', when 'bar' does not exist[1], svn copy foo bar@ and svn move foo bar@ don't create a destination file with the same name. (It's probably "bar@" that's the correct desination name in this case.) Can you please file an issue for this? Thanks Daniel Варфоломеев Игорь wrote on Tue, Jul 09, 2013 at 06:32:42 +0400: > Hi all, > > > > I think, I ran into a bug: > > > > although svn book declares “add one more @-sign” principle > > (http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html ), > > it looks like it’s working incorrectly in case of “svn move”. > > > > %svn% move wc\@2.txt@ wc\A\@2.txt@ > > > > - this would result in “wc\A\@2.txt@” file, instead of “wc\A\@2.txt”. > > > > > > It’s noteworthy, that “svn copy” works as it should. > > > > Please, find the full reproduction script here: > https://skydrive.live.com/redir?resid=8FD4EC3161ABA67A!939 . > > > > tested on svn, version 1.8.0 (r1490375); > > compiled Jun 17 2013, 19:47:41 on x86-microsoft-windows. > > (Part of Tortoise SVN 1.8.0 x64Win7 x64). > > > > > > PS > > I’m not subscribed and would appreciate being explicitly Cc:ed in any > responses. > > > > > > Best regards, > > Igor Varfolomeev > > >
RE: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
> -Original Message- > From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > Sent: Tuesday, July 09, 2013 1:22 PM > To: Andrew Reedick > Cc: users@subversion.apache.org > Subject: Re: "svn add" failing with "svn: E29: Can't set 'svn:eol- > style': ... has binary mime type property" and/or inconsistent newlines > > > Can you try setting "*.png =" in the auto-props section? Perhaps your > registry has a setting that that will override. That said, perhaps > your registry has a setting for "*.pn*" which that will _not_ override; > so check your registry: > >"REGISTRY:HKLM\\Software\\Tigris.org\\Subversion\\Config" >"REGISTRY:HKCU\\Software\\Tigris.org\\Subversion\\Config" > > For the archives, if this hadn't been a new repository, I'd have > suggested to look for inherited svn:auto-props properties too: > % svn pg --show-inherited-props -R svn:auto-props No joy. Setting "*.png" and enabling auto-props didn't work. The registry has no property settings (for both Tigris.org and Collabnet). And "svn pg --show-inherited-props -R svn:auto-props" returns nothing. However 'svn add --no-auto-props' does allow the add to work, but that's a bit drastic and inconvenient. That fact that 'svn add' fails on different files is really throwing me for a loop. I'm beginning to wonder if Something(tm) is borked with my workstation (anti-virus, some left-over DLL in the path, etc.)
Re: svnsync crashes on a huge commit
it's very unlikely to be a memory leak... svnsync worked for 20+ hours and synchronized 300 000+ revisions and downloaded 65+ Gb during the one sync session before it felt in this trap. now it stuck... and can't synchronize this revision after restart. The procedure is very slow, it took more than an hour to sync this particular revision with 56000 files in it. The cumulative files size in this revision is not very big, the size is less than 4gb. Before it crashes transaction folder grows to 46mb in size and txn-protorevs is 4Gb. Looks like it has already downloaded all the files and crashed after it. Looks like it is a problem in handling huge commit and not a memory leak, also may be the server terminated connection. On Tue, Jul 9, 2013 at 9:13 PM, Stefan Sperling wrote: > On Tue, Jul 09, 2013 at 03:30:44PM +0400, Anatoly Zapadinsky wrote: > > svnsync failed to sync repository with a single commit containing 56000 > > files. This commit handled perfectly by console svn and tortoise svn. We > > can checkout this folder structure and so on. But when I tried to sync > this > > commit to the local mirror repository it fails. > > 32 bit version of svnsync crashed with out of memory exception. > > 64 bit version made a transaction folder containing 117000 files in it, > > allocated like 3gb of memory and finally crashed with this diagnostic: > > "svn: E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: > > internal malfunction". > > May be this commit is not an example of the best practice but it crash > only > > svnsync all the other svn tools handle it perfectly. > > What should I do? How to mirror this repository? Is it a bug or not? > > This problem sounds like a memory leak. > I wonder if you're running into the same problem this user is seeing: > http://svn.haxx.se/users/archive-2013-07/0128.shtml >
RE: svnsync crashes on a huge commit
We've had similar problems with large files with our svnsync. It has always been a time out problem. If you are using httpd there is a time out value in httpd.conf that can be increased (I don't recall the exact parameter). We also use VIPs for our SVN servers and we had an issue with the VIP having a time out at 2 minutes. We had to increase that temporarily to accommodate some very large commits. Cheers, Thomas From: Anatoly Zapadinsky [mailto:zapadin...@gmail.com] Sent: Tuesday, July 09, 2013 1:48 PM To: Anatoly Zapadinsky; users@subversion.apache.org Subject: Re: svnsync crashes on a huge commit it's very unlikely to be a memory leak... svnsync worked for 20+ hours and synchronized 300 000+ revisions and downloaded 65+ Gb during the one sync session before it felt in this trap. now it stuck... and can't synchronize this revision after restart. The procedure is very slow, it took more than an hour to sync this particular revision with 56000 files in it. The cumulative files size in this revision is not very big, the size is less than 4gb. Before it crashes transaction folder grows to 46mb in size and txn-protorevs is 4Gb. Looks like it has already downloaded all the files and crashed after it. Looks like it is a problem in handling huge commit and not a memory leak, also may be the server terminated connection. On Tue, Jul 9, 2013 at 9:13 PM, Stefan Sperling mailto:s...@elego.de>> wrote: On Tue, Jul 09, 2013 at 03:30:44PM +0400, Anatoly Zapadinsky wrote: > svnsync failed to sync repository with a single commit containing 56000 > files. This commit handled perfectly by console svn and tortoise svn. We > can checkout this folder structure and so on. But when I tried to sync this > commit to the local mirror repository it fails. > 32 bit version of svnsync crashed with out of memory exception. > 64 bit version made a transaction folder containing 117000 files in it, > allocated like 3gb of memory and finally crashed with this diagnostic: > "svn: E235000: In file 'subversion/libsvn_ra_serf/util.c' line 1649: > internal malfunction". > May be this commit is not an example of the best practice but it crash only > svnsync all the other svn tools handle it perfectly. > What should I do? How to mirror this repository? Is it a bug or not? This problem sounds like a memory leak. I wonder if you're running into the same problem this user is seeing: http://svn.haxx.se/users/archive-2013-07/0128.shtml
Re: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
Andrew Reedick wrote on Tue, Jul 09, 2013 at 13:40:53 -0400: > However 'svn add --no-auto-props' does allow the add to work, but that's a > bit drastic and inconvenient. > > That fact that 'svn add' fails on different files is really throwing me for a > loop. I'm beginning to wonder if Something(tm) is borked with my workstation > (anti-virus, some left-over DLL in the path, etc.) Most likely you have an [auto-props] setting somewhere that's getting picked up.
Re: Check-out fails with LANG=C
On Tue, Jul 9, 2013 at 7:11 PM, Stefan Sperling wrote: > Can you please file a new issue pointing to this thread? Done. Issue 4391. - Michael
Re: Check-out fails with LANG=C
On 09.07.2013 17:01, Andreas Krey wrote: > On Tue, 09 Jul 2013 16:26:40 +, Branko ??ibej wrote: > ... >> Since we're on the topic, would you care to explain why translating >> files to the native encoding is "a rather stupid idea"? > Because LANG isn't "the native encoding", but, for the file system > the "naive encoding". > > What encoding is used in the file system is a property of the > file system or even individual directories and files, but certainly > not of the terminal session I'm using to do the checkout. > > (And if you think that LANG should apply to file names on disk, > why would you think it should not do so for the file names that > come from the svn server in the checkout?) Unlike on Windows and Mac OS (the latter at least with HFS+), the is no notion of native filesystem encoding on other Unix-like platforms. The best we can do is look at the locale settings, specifically, LC_CTYPE. I posit that if the "native encoding" is supposed to be UTF-8, then it is an error to use LANG=C at all. Instead, one should use LANG=C.UTF-8. In a context where, for example, most files were encoded in Big5 (http://en.wikipedia.org/wiki/Big5) — not a too far-fetched proposition — it would be slightly insane, to put it mildly, for Subversion to assume it can just write UTF-8 to disk. So indeed, this state of affairs puts the burden of setting up their locale correctly on users, but that's simply the way Unix works. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
RE: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
> -Original Message-
> From: Daniel Shahaf [mailto:d...@daniel.shahaf.name]
> Sent: Tuesday, July 09, 2013 2:00 PM
> To: Andrew Reedick
> Cc: users@subversion.apache.org
> Subject: Re: "svn add" failing with "svn: E29: Can't set 'svn:eol-
> style': ... has binary mime type property" and/or inconsistent newlines
>
> Andrew Reedick wrote on Tue, Jul 09, 2013 at 13:40:53 -0400:
> > However 'svn add --no-auto-props' does allow the add to work, but
> that's a bit drastic and inconvenient.
> >
> > That fact that 'svn add' fails on different files is really throwing
> > me for a loop. I'm beginning to wonder if Something(tm) is borked
> > with my workstation (anti-virus, some left-over DLL in the path,
> etc.)
>
> Most likely you have an [auto-props] setting somewhere that's getting
> picked up.
Bingo. "Somewhere" was the operative word. The Collabnet config file was
being read from my roaming profile instead of from my windows home dir. The
roaming copy of config had "* = svn:eol-style=LF", and that auto-prop seems to
have been causing the breakage. ("global-ignores = *.* *" is a great way to
narrow down what config file is being read.)
And the reason why 'svn add' was failing on different files is because 'svn
add' walks the directory tree in a different order each time.
Thanks for the help! Now I must find a wall to quietly beat my head against.
Re: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
On 09.07.2013 20:33, Andrew Reedick wrote: > Bingo. "Somewhere" was the operative word. The Collabnet config file was > being read from my roaming profile instead of from my windows home dir. You're aware that, on Windows, Subversion doesn't look for config files in %USERPROFILE%\.subversion at all but in %APPDATA%\Subversion? -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On Tue, 09 Jul 2013 20:21:33 +, Branko ??ibej wrote: ... > I posit that if the "native encoding" is supposed to be UTF-8, then it > is an error to use LANG=C at all. Instead, one should use LANG=C.UTF-8. No, using LANG etc. for the interface between svn and the disk mixes up the setup for the UI and the disk interface. LANG tells the software what language I want to use in the UI and how it should be encoded, and is intended to be able to talk in different ways depending on what terminal I use. Tying the svn disk interface to that value is obviously tying this property to the wrong object. ... > So indeed, this state of affairs puts the burden of setting up their > locale correctly on users, but that's simply the way Unix works. No, it ain't. It's almost like the mailer looks into LANG to see how to interpret the incoming mail, instead of using 'Charset:' etc. Needing to do 'LANG=C svn info' to be able to grep for keywords actually is the unix way, nowadays. en_us.utf8 is more appropriate nowadays, howevery I can't remember how it's spelled. Andreas -- "Totally trivial. Famous last words." From: Linus Torvalds Date: Fri, 22 Jan 2010 07:29:21 -0800
RE: "svn add" failing with "svn: E200009: Can't set 'svn:eol-style': ... has binary mime type property" and/or inconsistent newlines
> -Original Message- > From: Branko Čibej [mailto:br...@wandisco.com] > Sent: Tuesday, July 09, 2013 3:12 PM > To: users@subversion.apache.org > Subject: Re: "svn add" failing with "svn: E29: Can't set 'svn:eol- > style': ... has binary mime type property" and/or inconsistent newlines > > On 09.07.2013 20:33, Andrew Reedick wrote: > > Bingo. "Somewhere" was the operative word. The Collabnet config > file was being read from my roaming profile instead of from my windows > home dir. > > You're aware that, on Windows, Subversion doesn't look for config files > in %USERPROFILE%\.subversion at all but in %APPDATA%\Subversion? > I'm aware now. (I've been using a Cygwin svn client for some time now, which reads from ~/.subversion.) The real question is why I had a %USERPROFILE%\.subversion tree in the first place (timestamps are from January.) *shrug*
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 09:22:39PM +0200, Andreas Krey wrote: > On Tue, 09 Jul 2013 20:21:33 +, Branko ??ibej wrote: > ... > > I posit that if the "native encoding" is supposed to be UTF-8, then it > > is an error to use LANG=C at all. Instead, one should use LANG=C.UTF-8. > > No, using LANG etc. for the interface between svn and the disk mixes up > the setup for the UI and the disk interface. LANG tells the software > what language I want to use in the UI and how it should be encoded, > and is intended to be able to talk in different ways depending on what > terminal I use. LANG implies LC_CTYPE, if not set. svn prefers LC_CTYPE over LANG. LC_CTYPE defines parts of the C runtime behaviour. It's not a UI knob the user sets willy-nilly. http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html There is no standardized way of figuring out filename encoding on UNIX, unfortunately. Subversion has chosen one way of working around that by looking at locale configuration. The other workaround is to always use UTF-8 and hope users and other applications can cope with that. I think using UTF-8 by default would be a good choice today. But it certainly wasn't when the Subversion project was started years ago. And we cannot change the existing default behaviour now. That would create compatibility nightmares with existing working copies. I don't really understand what kind of answers you are hoping to get. Are you actually proposing that we change something in Subversion, or are you arguing out of spite?
Re: Check-out fails with LANG=C
On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: ... > I think using UTF-8 by default would be a good choice today. But it > certainly wasn't when the Subversion project was started years ago. > And we cannot change the existing default behaviour now. That would > create compatibility nightmares with existing working copies. You could still make the encoding settable in the WC configuration, and optionally make that default to utf8 in the checkout. Old WCs wouldn't be affected. If the filesystem doesn't know, the application should store a value. (This whole stuff is scary anyway.) > I don't really understand what kind of answers you are hoping to get. 'You might have a point there'? > Are you actually proposing that we change something in Subversion, > or are you arguing out of spite? False dichotomy? My spite is currently reserved for clearcase. Andreas -- "Totally trivial. Famous last words." From: Linus Torvalds Date: Fri, 22 Jan 2010 07:29:21 -0800
Re: Check-out fails with LANG=C
On 09.07.2013 21:22, Andreas Krey wrote: >> So indeed, this state of affairs puts the burden of setting up their >> locale correctly on users, but that's simply the way Unix works. > No, it ain't. It's almost like the mailer looks into LANG to see > how to interpret the incoming mail, instead of using 'Charset:' etc. Bad analogy, I'm afraid. It's a bit like the mailer looking at LC_CTYPE to determine which encoding to use when /sending/ mails. Which would be reasonable. In any case, as I said before, Unix filesystems do not standardize a file name encoding. One has to assume that the locale is set correctly, or be incompatible with all other applications running on the system. This is not a new problem, it has existed since the first time someone tried to use a non-ASCII encoding on a Unix system. > Needing to do 'LANG=C svn info' to be able to grep for keywords > actually is the unix way, nowadays. en_us.utf8 is more appropriate > nowadays, howevery I can't remember how it's spelled. Using "LANG=C" is exactly wrong, as I said earlier, unless you know that you only need to deal with file names in the ASCII subset. Given that we're talking about Subversion, where we know that the internal encoding is always UTF-8, it's dangerous to make that assumption. If you want to be pedantic, you should use "LC_MESSAGES=C LC_CTYPE=UTF-8", which is roughly equivalent in the context of this discussion to the "LANG=C.UTF-8" that I proposed a couple of posts ago. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On 09.07.2013 21:43, Stefan Sperling wrote: > On Tue, Jul 09, 2013 at 09:22:39PM +0200, Andreas Krey wrote: >> On Tue, 09 Jul 2013 20:21:33 +, Branko ??ibej wrote: >> ... >>> I posit that if the "native encoding" is supposed to be UTF-8, then it >>> is an error to use LANG=C at all. Instead, one should use LANG=C.UTF-8. >> No, using LANG etc. for the interface between svn and the disk mixes up >> the setup for the UI and the disk interface. LANG tells the software >> what language I want to use in the UI and how it should be encoded, >> and is intended to be able to talk in different ways depending on what >> terminal I use. > LANG implies LC_CTYPE, if not set. svn prefers LC_CTYPE over LANG. Specifically, Subversion's command-line tools first try setlocale(LC_ALL, "") and if that fails, setlocale(LC_CTYPE, "") Only if the latter fails, we look at the actual environment to construct an error message. So we don't use environment variables to determine which translation and encoding to use, we rely on the locale API in the runtime library. This is the only correct way for command-line programs to determine the locale-related characteristics of the system they're running on. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On 09.07.2013 22:02, Andreas Krey wrote: > On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: > ... > >> I don't really understand what kind of answers you are hoping to get. > 'You might have a point there'? Why should anyone concede your point when all cited documentation says otherwise? -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 10:02:00PM +0200, Andreas Krey wrote: > On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: > ... > > I think using UTF-8 by default would be a good choice today. But it > > certainly wasn't when the Subversion project was started years ago. > > And we cannot change the existing default behaviour now. That would > > create compatibility nightmares with existing working copies. > > You could still make the encoding settable in the WC configuration, > and optionally make that default to utf8 in the checkout. Old WCs > wouldn't be affected. If the filesystem doesn't know, the application > should store a value. (This whole stuff is scary anyway.) Ok, fine. I can see that working and be useful. Apart from the fact that we currently do not really have a concept of a per-WC configuration. But with the default set to the old behaviour please. We don't want to cause surprises for unsuspecting users. This knob would have to be in the client config, I guess, and apply to all working copies during checkout and update. The working copy would of course need to remember the value of the config knob had during the initial checkout (this can be stored in wc.db), and would re-use that value even if the configuration knob is changed. What if the encoding specified in the config is not supported by the iconv library? Do we fall back to LANG or ASCII? Re-encoding all filenames in a working copy would only be possible via a new 'svn checkout', I guess. Same as resetting LANG and running 'svn checkout' again. I recall us having had this discussion before, by the way: http://svn.haxx.se/users/archive-2011-06/0296.shtml Another idea: Introduce a new inheritable property that dictates what encoding clients should be using for filenames. This would probably be very easy to mess up when multiple client platforms are involved, but could prevent problems for users who don't set up their config or LANG correctly where admins just want this to work out of the box within their own organisation. > > I don't really understand what kind of answers you are hoping to get. > > 'You might have a point there'? You do. Now please file an issue or send a patch :) > > Are you actually proposing that we change something in Subversion, > > or are you arguing out of spite? > > False dichotomy? My spite is currently reserved for clearcase. Ouch. I've seen others feel your pain :( It's not very pleasant indeed.
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 10:27:29PM +0200, Branko Čibej wrote: > In any case, as I said before, Unix filesystems do not standardize a > file name encoding. One has to assume that the locale is set correctly, > or be incompatible with all other applications running on the system. > This is not a new problem, it has existed since the first time someone > tried to use a non-ASCII encoding on a Unix system. But that doesn't mean that we couldn't supply a configuration based approach for specifying the encoding to use, for those users who'd really want that. As long as any such feature leaves the existing default behaviour alone, I don't see an issue. Now, I still wonder why anyone would want to mix and match encodings on their filesystems. But this isn't the first time this issue has been brought up, so it seems to be important to some of our users.
Re: Check-out fails with LANG=C
On Jul 9, 2013, at 15:47, Stefan Sperling wrote: > On Tue, Jul 09, 2013 at 10:02:00PM +0200, Andreas Krey wrote: >> On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: >> ... >>> I think using UTF-8 by default would be a good choice today. But it >>> certainly wasn't when the Subversion project was started years ago. >>> And we cannot change the existing default behaviour now. That would >>> create compatibility nightmares with existing working copies. >> >> You could still make the encoding settable in the WC configuration, >> and optionally make that default to utf8 in the checkout. Old WCs >> wouldn't be affected. If the filesystem doesn't know, the application >> should store a value. (This whole stuff is scary anyway.) > > Ok, fine. I can see that working and be useful. Apart from the fact > that we currently do not really have a concept of a per-WC configuration. > > But with the default set to the old behaviour please. We don't want > to cause surprises for unsuspecting users. > > This knob would have to be in the client config, I guess, and apply > to all working copies during checkout and update. The working copy > would of course need to remember the value of the config knob had > during the initial checkout (this can be stored in wc.db), and would > re-use that value even if the configuration knob is changed. What happens if you copy a working copy between filesystems that have different encodings? What happens if you copy a working copy from OS X or Windows (whose default filesystems already know the disk encoding) to Linux or other UNIX (whose popular filesystems don't), or vice versa?
Re: Check-out fails with LANG=C
On 09.07.2013 22:47, Stefan Sperling wrote: > On Tue, Jul 09, 2013 at 10:02:00PM +0200, Andreas Krey wrote: >> On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: >> ... >>> I think using UTF-8 by default would be a good choice today. But it >>> certainly wasn't when the Subversion project was started years ago. >>> And we cannot change the existing default behaviour now. That would >>> create compatibility nightmares with existing working copies. >> You could still make the encoding settable in the WC configuration, >> and optionally make that default to utf8 in the checkout. Old WCs >> wouldn't be affected. If the filesystem doesn't know, the application >> should store a value. (This whole stuff is scary anyway.) > Ok, fine. I can see that working and be useful. Apart from the fact > that we currently do not really have a concept of a per-WC configuration. Sorry, I have to ask, useful *how* exactly? There is already a perfectly valid workaround for the case that Andreas is talking about, namely, relieving scripts authors from the complexity of parsing multi-lingual responses. It's just that that perfectly valid workaround happens to not be "env LANG=C". Why invent another configuration knob that you already know (given the rest of your post) is going to cause all sorts of trouble? -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On 09.07.2013 22:53, Ryan Schmidt wrote: > On Jul 9, 2013, at 15:47, Stefan Sperling wrote: >> On Tue, Jul 09, 2013 at 10:02:00PM +0200, Andreas Krey wrote: >>> On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: >>> ... I think using UTF-8 by default would be a good choice today. But it certainly wasn't when the Subversion project was started years ago. And we cannot change the existing default behaviour now. That would create compatibility nightmares with existing working copies. >>> You could still make the encoding settable in the WC configuration, >>> and optionally make that default to utf8 in the checkout. Old WCs >>> wouldn't be affected. If the filesystem doesn't know, the application >>> should store a value. (This whole stuff is scary anyway.) >> Ok, fine. I can see that working and be useful. Apart from the fact >> that we currently do not really have a concept of a per-WC configuration. >> >> But with the default set to the old behaviour please. We don't want >> to cause surprises for unsuspecting users. >> >> This knob would have to be in the client config, I guess, and apply >> to all working copies during checkout and update. The working copy >> would of course need to remember the value of the config knob had >> during the initial checkout (this can be stored in wc.db), and would >> re-use that value even if the configuration knob is changed. > What happens if you copy a working copy between filesystems that have > different encodings? > > What happens if you copy a working copy from OS X or Windows (whose default > filesystems already know the disk encoding) to Linux or other UNIX (whose > popular filesystems don't), or vice versa? That's largely off-topic here because it has nothing to do with Subversion as such; the short answer is, it depends on how you're doing the copying, which network protocol you use, and how it's set up. For example, if you're doing this via a Samba server running on the Linux box, the file names will be re-encoded to whatever you configured Samba to use. If that's different from what your target system expects, you're in trouble. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On 09.07.2013 22:50, Stefan Sperling wrote: > On Tue, Jul 09, 2013 at 10:27:29PM +0200, Branko Čibej wrote: >> In any case, as I said before, Unix filesystems do not standardize a >> file name encoding. One has to assume that the locale is set correctly, >> or be incompatible with all other applications running on the system. >> This is not a new problem, it has existed since the first time someone >> tried to use a non-ASCII encoding on a Unix system. > But that doesn't mean that we couldn't supply a configuration > based approach for specifying the encoding to use, for those > users who'd really want that. As long as any such feature leaves > the existing default behaviour alone, I don't see an issue. I do. It's another feature to support, another set of interactions to worry about, and another source of ambiguities in bug reports. And all that, I'll stress again, for no good reason. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 10:53:41PM +0200, Branko Čibej wrote: > Sorry, I have to ask, useful *how* exactly? There is already a perfectly > valid workaround for the case that Andreas is talking about, namely, > relieving scripts authors from the complexity of parsing multi-lingual > responses. It's just that that perfectly valid workaround happens to not > be "env LANG=C". This is not about error messages. There is another, unrelated problem, that was brought up here: http://svn.haxx.se/users/archive-2011-06/0293.shtml Namely, if users change locales, they cannot update a working copy without getting errors or filenames with mixed encodings. Users might have to change locales e.g. when working remotely on a system that doesn't support the initially chosen locale, or when working with a text console that doesn't support, say, UTF-8. > Why invent another configuration knob that you already know (given the > rest of your post) is going to cause all sorts of trouble? I think in this case, if users use the knob and run into trouble, they get to keep the pieces. We'd just be giving them a way to use a working copy with a specific filename encoding independenly of the currently active lcoale. And I'm not going to go out of my way for adding such a feature. All I'm saying is that I wouldn't object to it in principle.
Re: Check-out fails with LANG=C
On Jul 9, 2013, at 15:59, Branko Čibej wrote: > On 09.07.2013 22:53, Ryan Schmidt wrote: >> What happens if you copy a working copy between filesystems that have >> different encodings? >> >> What happens if you copy a working copy from OS X or Windows (whose default >> filesystems already know the disk encoding) to Linux or other UNIX (whose >> popular filesystems don't), or vice versa? > > That's largely off-topic here because it has nothing to do with > Subversion as such; the short answer is, it depends on how you're doing > the copying, which network protocol you use, and how it's set up. > > For example, if you're doing this via a Samba server running on the > Linux box, the file names will be re-encoded to whatever you configured > Samba to use. If that's different from what your target system expects, > you're in trouble. I just meant that if you record inside the working copy the encoding of the filesystem you're checking out on, that might not be the same as the encoding of the filesystem you might later copy that working copy to.
Re: Check-out fails with LANG=C
On Tue, Jul 09, 2013 at 04:10:09PM -0500, Ryan Schmidt wrote: > I just meant that if you record inside the working copy the encoding of the > filesystem you're checking out on, that might not be the same as the encoding > of the filesystem you might later copy that working copy to. In case the copy command re-encodes the filenames, the encoding stored in the working copy encoding might be different to the one used by filenames, indeed.
RE: "svn add *" and sign in filename
From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > > Run 'svn add wc/b...@2.txt@'. Did you mean "svn add wc\B\@2.txt@" ? The latter, defiantly, would work, but I'd like to add all files and folders inside folder "B" instead. And I found that whether %svn% add wc\B\* command would work depends on whether there's a file with "@" character inside.
Re: "svn add *" and sign in filename
Варфоломеев Игорь wrote on Wed, Jul 10, 2013 at 04:17:37 +0400: > From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > > > > Run 'svn add wc/b...@2.txt@'. > > Did you mean "svn add wc\B\@2.txt@" ? Forward and backward slashes are interchangeable when using svn on windows. > The latter, defiantly, would work, but I'd like to add all files and folders > inside folder "B" instead. > > And I found that whether > %svn% add wc\B\* Just do: svn add wc\B (if B is alreadyed versioned, add --force to that command)
RE: "svn add *" and sign in filename
> From: 'Daniel Shahaf' [mailto:d...@daniel.shahaf.name] > Sent: Wednesday, July 10, 2013 4:25 AM > > Варфоломеев Игорь wrote on Wed, Jul 10, 2013 at 04:17:37 +0400: > > From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > > > > > > Run 'svn add wc/b...@2.txt@'. > > > > Did you mean "svn add wc\B\@2.txt@" ? > > Forward and backward slashes are interchangeable when using svn on > windows. They are... I've only meant that "svn add wc/b...@2.txt@" lacks a slash after "B"... The interesting thing is - this is exactly how it's displayed in an error message: > In particular > svn add wc\B\* > results in > svn: E29: 'wc/b...@2.txt': a peg revision is not allowed here if > there's a file "@2.txt" inside "B". > Just do: svn add wc\B > > (if B is alreadyed versioned, add --force to that command) This works, yep. But this looks more like a workaround (in case "B" itself is already versioned). I don't say this is a critical bug, but I think if "svn add wc\B\*" syntax is working in one case, It should work in all cases, independently of directory contents
Re: "svn add *" and sign in filename
Варфоломеев Игорь wrote on Wed, Jul 10, 2013 at 04:39:23 +0400: > > From: 'Daniel Shahaf' [mailto:d...@daniel.shahaf.name] > > Sent: Wednesday, July 10, 2013 4:25 AM > > > > Варфоломеев Игорь wrote on Wed, Jul 10, 2013 at 04:17:37 +0400: > > > From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > > > > > > > > Run 'svn add wc/b...@2.txt@'. > > > > > > Did you mean "svn add wc\B\@2.txt@" ? > > > > Forward and backward slashes are interchangeable when using svn on > > windows. > > They are... I've only meant that "svn add wc/b...@2.txt@" lacks a slash after > "B"... > The interesting thing is - this is exactly how it's displayed in an error > message: > > > > In particular > > svn add wc\B\* > > results in > > svn: E29: 'wc/b...@2.txt': a peg revision is not allowed here if > > there's a file "@2.txt" inside "B". The glob is expanded by the shell, so svn sees ['svn', 'add', 'wc/B/@2.txt'], so it parses out target 'wc/B/' with peg '2.txt' (this part is arguably a bug), and stripping the slash off the end is normal (see svn_dirent_canonicalize() in if you're curious). > > > > Just do: svn add wc\B > > > > (if B is alreadyed versioned, add --force to that command) > > This works, yep. But this looks more like a workaround (in case "B" itself is > already versioned). > I don't say this is a critical bug, but I think if "svn add wc\B\*" syntax is > working in one case, > It should work in all cases, independently of directory contents Globs are expanded by the shell, so that's just another facet of the original bug, whereby 'svn add' requires escaping '@' signs in filenames. Feel free to file an issue about that, so we don't forget it. (But please check first if there is, or was, another issue for this.)
RE: svn move and sign in filepath
> From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > Sent: Tuesday, July 09, 2013 9:30 PM > > Can you please file an issue for this? OK. I've just created the issue: http://subversion.tigris.org/issues/show_bug.cgi?id=4392 Thank you! > (It's probably "bar@" that's the correct desination name in this case.) Em... is it? I thought, " svn cares only about the last at sign in the argument, and it is not considered illegal to omit a literal peg revision specifier after that at sign. " ( http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html ) means that only "@" in "svn copy foo bar@" is considered as an "at-sign for revision".
Re: svn move and sign in filepath
Варфоломеев Игорь wrote on Wed, Jul 10, 2013 at 05:22:49 +0400: > > From: Daniel Shahaf [mailto:d...@daniel.shahaf.name] > > Sent: Tuesday, July 09, 2013 9:30 PM > > > > > Can you please file an issue for this? > > OK. I've just created the issue: > http://subversion.tigris.org/issues/show_bug.cgi?id=4392 > Thank you! > > > > (It's probably "bar@" that's the correct desination name in this case.) > > Em... is it? > I thought, " svn cares only about the last at sign in the argument, and it is > not considered > illegal to omit a literal peg revision specifier after that at sign. " > ( http://svnbook.red-bean.com/en/1.7/svn.advanced.pegrevs.html ) > means that only "@" in "svn copy foo bar@" is considered as an "at-sign for > revision". The rule for parsing a peg revision from a string is: if '@' is present, everything up to and not including the rightmost '@' is the path, and everything to the left of the rightmost '@' is the peg; else, the peg is unspecified. As it happens, an empty string after the rightmost @ is valid. That's why appending an '@' to a filename escapes it (regardless of whether the filename includes an '@' sign or not). The question is whether we should not be parsing peg revisions in some places. The target arguments to 'add' seem like a clear-cut case. The 'dest' argument of move seems like such a case too: assuming Q is a versioned directory, what would be the meaning of, say, 'svn move alpha beta gamma Q@r42'?
Re: Check-out fails with LANG=C
On Tue, Jul 9, 2013 at 4:53 PM, Ryan Schmidt wrote: > On Jul 9, 2013, at 15:47, Stefan Sperling wrote: >> On Tue, Jul 09, 2013 at 10:02:00PM +0200, Andreas Krey wrote: >>> On Tue, 09 Jul 2013 21:43:50 +, Stefan Sperling wrote: >>> ... I think using UTF-8 by default would be a good choice today. But it certainly wasn't when the Subversion project was started years ago. And we cannot change the existing default behaviour now. That would create compatibility nightmares with existing working copies. >>> >>> You could still make the encoding settable in the WC configuration, >>> and optionally make that default to utf8 in the checkout. Old WCs >>> wouldn't be affected. If the filesystem doesn't know, the application >>> should store a value. (This whole stuff is scary anyway.) >> >> Ok, fine. I can see that working and be useful. Apart from the fact >> that we currently do not really have a concept of a per-WC configuration. >> >> But with the default set to the old behaviour please. We don't want >> to cause surprises for unsuspecting users. >> >> This knob would have to be in the client config, I guess, and apply >> to all working copies during checkout and update. The working copy >> would of course need to remember the value of the config knob had >> during the initial checkout (this can be stored in wc.db), and would >> re-use that value even if the configuration knob is changed. > > What happens if you copy a working copy between filesystems that have > different encodings? > > What happens if you copy a working copy from OS X or Windows (whose default > filesystems already know the disk encoding) to Linux or other UNIX (whose > popular filesystems don't), or vice versa? You hve *precisely* the same kind of problem with svn:eol settings.
xml output changed - relative paths now appearing?
Greetings all, My trouble is that the XML output seems to have changed with the 1.8.0 subversion library. My interaction with Subversion is via the TortoiseSVN software. (I have never emailed this group - I am completely willing to rephrase my question and/or pursue additional homework. I did review the open issues, the Docs, and the FAQ before emailing this. I even rummaged around in both the TortoiseSVN and the Subversion SVN Logs - to see if I might have beginner's luck to spot some commit which explained this change, but alas I didn't find anything fruitful). So, using TortoiseSVN 1.7.11 (linked against SVN 1.7.8) , with the command line utilities installed , from a CMD (dos shell) window , do the following: CD\Program Files\TortoiseSVN\Bin svn status --verbose --xml C:\PATH\TO\REPOSITORY then , using TortoiseSVN 1.8.0 (aka linked against SVN 1.8.0) repeat the test, and compare the xml outputs. For me (on a Win XP 32bit machine), they differ in a very key fashion: "old" xml (just a leading excerpt) * * ... ... THE KEY bit is the PATH="" parts . notice how they specific a full absolute path? Then, contrast to the "new" xml output: * * ... ... NOW SEE how the target and entry paths differ. The entry path possess enough ..\ "parent" folder references to "dig out of the" Program Files\TortoiseSVN\bin path all the way back up to the point from which the \svnfocus\versions\version8\dev path might be constructed. I've experimented moving the execution directory around - i.e. executing the svn command from C:\ or C:\OTHER\FOLDER\PATH , and the ..\..\ changes accordingly. Any advice or followup requests? Thanks, Alexander -- Alexander Haley, Computer Scientist, 781-774-5156 Medical Information Technology, Inc. Mailstop: F4E16W, MEDITECH Circle, Westwood, MA 02090
dav_svn__get_inherited_props_report and NetBSD 5.2 install
G'day, I'm trying to get subversion 1.8 up on a NetBSD 5.2 (amd64) box using pkgsrc, and am seeing the following at run-time : bash-4.2# /etc/rc.d/apache start Starting apache. httpd: Syntax error on line 166 of /usr/pkg/etc/httpd/httpd.conf: Cannot load libexec/mod_dav_svn.so into server: /usr/pkg/libexec/mod_dav_svn.so: Undefined PLT symbol "dav_svn__get_inherited_props_report" (symnum = 421) I've asked the pkgsrc people and they're of the opinion that this is a subversion issue not a pkgsrc issue, so here I am. I've grepped through the source code and found dav_svn__get_inherited_props_report in three files : subversion-1.8.0/subversion/mod_dav_svn/dav_svn.h subversion-1.8.0/subversion/mod_dav_svn/reports/inherited-props.c subversion-1.8.0/subversion/mod_dav_svn/version.c and if I check the freshly built mod_dav_svn.so : bash-4.2# strings /usr/pkg/libexec/mod_dav_svn.so | grep dav_svn__get_inherited_props_report dav_svn__get_inherited_props_report It's there. Any ideas as to why I'm seeing that error? Please CC to me if possible. Thank you Carl
Re: xml output changed - relative paths now appearing?
Guten Tag Alexander Haley, am Dienstag, 9. Juli 2013 um 21:20 schrieben Sie: > Any advice or followup requests? What do you need the paths for? If it's by design that they are relative now to the "current directory" you may have no other choice than to either calculate absolute paths on your own using the execution directory of Tortoise's svn cmdline clients or simply execute those clients on your own with in a directory which produces better/easier relative paths for you. Mit freundlichen Grüßen, Thorsten Schöning -- Thorsten Schöning E-Mail:thorsten.schoen...@am-soft.de AM-SoFT IT-Systeme http://www.AM-SoFT.de/ Telefon...05151- 9468- 55 Fax...05151- 9468- 88 Mobil..0178-8 9468- 04 AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
