Error processing externals E205011 and W155035 with simple update

2013-08-09 Thread dlellis
Hello,

I am running SVN 1.8.1 under Win7.   We have an extensive collection of 
file externals that we use to manage shared code between projects 
(hundreds of files per working directory).

As known, the performance when performing an SVN Update is slow.  To work 
around this, we're attempting to only update externals that are out of 
date (by comparing the local revisions of each file to what is in the 
svn:externals.  As we iterate across the files, if we see an external out 
of date, we execute an svn update . 

Doing this, I'm starting to get corrupted externals  I can reproduce 
this using the following the steps listed below.  Even if I am performing 
operations out of order with what is expected, I'd expect that I'd be 
unable to harm or corrupt the working directory


I basically create a clean checkout of a repository.
Verify everything is cool, but realize one file external needs a new 
revision. 
I edit the property (using TSVN) and commit the property.
I then update the working directory ignoring externals and only for the 
immediate directory (because external updates are slow).
I then update the file I know is out of date (because I just edited the 
svn;externals).
Ouch, I get an error and it reports that the file has been switched.
The only way I can repair this is to delete the checkout and recreate it.

I believe this also occurs under 1.7, but will have to confirm later.

Can any confirm first that this is indeed an issue, and second how to 
resolve this?

Thanks!
Dan


c:\Project_files\Test>svn status -v --depth immediates
      2850 2850 dlellis  .
X     2818 2818 dlellis  1.txt
X     2633 2633 dlellis  6.txt
X     2626 2626 dlellis  7.txt
      2850 2847 dlellis  CLP
      2850 2783 dlellis  SM
X     2812 2812 dlellis  checkintest.txt
X     2814 2814 dlellis  commit.vbs
X     2728 2728 dlellis  project.conf

c:\Project_files\Test>svn pg svn:externals
http://test/pd_sandbox/plcm/swdb/M4000/project.conf@2728 project.conf
http://test/pd_sandbox/plcm/swdb/Temp/6.txt@2633 6.txt
http://test/pd_sandbox/plcm/swdb/Temp/7.txt@2626 7.txt
http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2818 1.txt
http://test/pd_sandbox/plcm/swdb/Temp/commit.vbs@2814 commit.vbs
http://test/pd_sandbox/plcm/swdb/Temp/checkintest.txt@2812 checkintest.txt

* (Modify external here via TSVN) 1.txt from revision 2818 to 2827.

c:\Project_files\Test>svn pg svn:externals
http://test/pd_sandbox/plcm/swdb/M4000/project.conf@2728 project.conf
http://test/pd_sandbox/plcm/swdb/Temp/6.txt@2633 6.txt
http://test/pd_sandbox/plcm/swdb/Temp/7.txt@2626 7.txt
http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2827 1.txt
http://test/pd_sandbox/plcm/swdb/Temp/commit.vbs@2814 commit.vbs
http://test/pd_sandbox/plcm/swdb/Temp/checkintest.txt@2812 checkintest.txt

c:\Project_files\Test>svn commit . --depth empty -m "checkin test"
Sending.
Committed revision 2851.

c:\Project_files\Test>svn update --depth empty --ignore-externals
Updating '.':
At revision 2851.

c:\Project_files\Test>svn update 1.txt
Updating '1.txt':
U1.txt
Updated to revision 2851.

c:\Project_files\Test>svn update
Fetching external item into '1.txt':
***svn: warning: W155035: The specified path has an unexpected status
At revision 2851.
*** svn: E205011: Failure occurred processing one or more externals 
definitions

c:\Project_files\Test>svn status -v --depth immediates
  2851 2851 dlellis  .
***S 2851 2827 dlellis      1.txt
X 2633 2633 dlellis      6.txt
X 2626 2626 dlellis  7.txt
  2851 2847 dlellis  CLP
  2851 2783 dlellis  SM
    X     2812 2812 dlellis  checkintest.txt
    X 2814 2814 dlellis  commit.vbs
X 2728 2728 dlellis  project.conf


How to change paths on an external file without a full update --depth infinity?

2013-08-09 Thread dlellis
Hello everyone,

In an attempt to work around the slow performance issues with externals, 
I'm trying to perform selective updates on external files without 
performing an "svn update --depth infinity".

If I update the path on an external for foo.c to be from /bar1/ to /bar2/, 
and commit the property change, I'd hope that an "svn update foo.c" would 
result in foo.c being changed from /bar1/ to /bar2/, but it gets deleted 
instead.  I understand why foo.c gets deleted (/bar1/foo.c is different 
pedigree than /bar2/foo.c despite the same file name), but would really 
hope to perform an "svn update foo.c" to get the latest version referenced 
by the svn:externals.

How can I get svn to fetch me the file referenced in svn:externals without 
performing an "svn update --depth infinity"? 

Remind me again why externals are only processed when depth is infinity?

Thanks!
Dan



Re: Error processing externals E205011 and W155035 with simple update

2013-08-09 Thread dlellis
Hello everyone,

I have confirmed that this behavior also exists in SVN 1.7.5 on Win7.  I 
appreciate any thoughts any one might have.

Thanks,
Dan




From:   dlel...@rockwellcollins.com
To: users@subversion.apache.org
Date:   08/09/2013 03:55 PM
Subject:Error processing externals E205011 and W155035 with simple 
update



Hello, 

I am running SVN 1.8.1 under Win7.   We have an extensive collection of 
file externals that we use to manage shared code between projects 
(hundreds of files per working directory). 

As known, the performance when performing an SVN Update is slow.  To work 
around this, we're attempting to only update externals that are out of 
date (by comparing the local revisions of each file to what is in the 
svn:externals.  As we iterate across the files, if we see an external out 
of date, we execute an svn update . 

Doing this, I'm starting to get corrupted externals  I can reproduce 
this using the following the steps listed below.  Even if I am performing 
operations out of order with what is expected, I'd expect that I'd be 
unable to harm or corrupt the working directory 


I basically create a clean checkout of a repository. 
Verify everything is cool, but realize one file external needs a new 
revision.   
I edit the property (using TSVN) and commit the property. 
I then update the working directory ignoring externals and only for the 
immediate directory (because external updates are slow). 
I then update the file I know is out of date (because I just edited the 
svn;externals). 
Ouch, I get an error and it reports that the file has been switched. 
The only way I can repair this is to delete the checkout and recreate it. 

I believe this also occurs under 1.7, but will have to confirm later. 

Can any confirm first that this is indeed an issue, and second how to 
resolve this? 

Thanks! 
Dan 


c:\Project_files\Test>svn status -v --depth immediates 
      2850 2850 dlellis  . 
X     2818 2818 dlellis  1.txt 
X     2633 2633 dlellis  6.txt 
X     2626 2626 dlellis  7.txt 
      2850 2847 dlellis  CLP 
      2850 2783 dlellis  SM 
X     2812 2812 dlellis  checkintest.txt 
X     2814 2814 dlellis  commit.vbs 
X     2728 2728 dlellis  project.conf 
c:\Project_files\Test>svn pg svn:externals 
http://test/pd_sandbox/plcm/swdb/M4000/project.conf@2728 project.conf 
http://test/pd_sandbox/plcm/swdb/Temp/6.txt@2633 6.txt 
http://test/pd_sandbox/plcm/swdb/Temp/7.txt@2626 7.txt 
http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2818 1.txt 
http://test/pd_sandbox/plcm/swdb/Temp/commit.vbs@2814 commit.vbs 
http://test/pd_sandbox/plcm/swdb/Temp/checkintest.txt@2812 checkintest.txt 

* (Modify external here via TSVN) 1.txt from revision 2818 to 2827. 
c:\Project_files\Test>svn pg svn:externals 
http://test/pd_sandbox/plcm/swdb/M4000/project.conf@2728 project.conf 
http://test/pd_sandbox/plcm/swdb/Temp/6.txt@2633 6.txt 
http://test/pd_sandbox/plcm/swdb/Temp/7.txt@2626 7.txt 
http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2827 1.txt 
http://test/pd_sandbox/plcm/swdb/Temp/commit.vbs@2814 commit.vbs 
http://test/pd_sandbox/plcm/swdb/Temp/checkintest.txt@2812 checkintest.txt 

c:\Project_files\Test>svn commit . --depth empty -m "checkin test" 
Sending. 
Committed revision 2851. 
c:\Project_files\Test>svn update --depth empty --ignore-externals 
Updating '.': 
At revision 2851. 
c:\Project_files\Test>svn update 1.txt 
Updating '1.txt': 
U1.txt 
Updated to revision 2851. 
c:\Project_files\Test>svn update 
Fetching external item into '1.txt': 
***svn: warning: W155035: The specified path has an unexpected status 
At revision 2851. 
*** svn: E205011: Failure occurred processing one or more externals 
definitions 
c:\Project_files\Test>svn status -v --depth immediates 
  2851 2851 dlellis  . 
***S 2851 2827 dlellis  1.txt 
X 2633 2633 dlellis  6.txt 
X 2626 2626 dlellis      7.txt 
  2851 2847 dlellis  CLP 
  2851 2783 dlellis  SM 
    X 2812 2812 dlellis  checkintest.txt 
X 2814 2814 dlellis  commit.vbs 
X 2728 2728 dlellis  project.conf



Re: How to change paths on an external file without a full update --depth infinity?

2013-08-10 Thread dlellis
Hi Johan,

I think you are spot on on both issues - thank you.

The performance issue relates to SVN externals stem, in my case, from new 
connections for each http request for externals versus keeping an open one 
for non-externals.  Each connection results in a second or so delay. 
Multiply this by hundreds and updates that would take seconds, now takes 
ten minutes.  There's been quite a bit of discussion about pooling 
connections I believe, but I don't know the current state of this effort 
or if its in any plan.

We're attempting to work around this by updating the subversion properties 
with  svn update . --depth empty --ignore-externals and iterating over the 
externals and comparing to our local revision.  If we see a difference, we 
update the file.  This is the behavior I'd like to see externals take in 
the end (I'm sure there are good reasons for the current methodology). 
This issue and another issue that can corrupt the working copy are 
creating some barriers for us.

Externals are becoming more and more important as we look to finer grained 
product line sharing of code. 

All that said, is there a work around to convincing SVN to recreate the 
file without a full fledge update?

Thanks,
Dan




From:   Johan Corveleyn 
To: dlel...@rockwellcollins.com
Cc: "users@subversion.apache.org" 
Date:   08/10/2013 03:15 PM
Subject:Re: How to change paths on an external file without a full 
update --depth infinity?



On Sat, Aug 10, 2013 at 1:07 AM,   wrote:
> Hello everyone,
>
> In an attempt to work around the slow performance issues with externals, 
I'm
> trying to perform selective updates on external files without performing 
an
> "svn update --depth infinity".

First: what "slow performance issues with externals"? Can you give
some concrete numbers, or some references to other posts about this?

What svn version are you using? Is this slowness a new problem?

> If I update the path on an external for foo.c to be from /bar1/ to 
/bar2/,
> and commit the property change, I'd hope that an "svn update foo.c" 
would
> result in foo.c being changed from /bar1/ to /bar2/, but it gets deleted
> instead.  I understand why foo.c gets deleted (/bar1/foo.c is different
> pedigree than /bar2/foo.c despite the same file name), but would really 
hope
> to perform an "svn update foo.c" to get the latest version referenced by 
the
> svn:externals.

This looks similar to this issue:

  http://subversion.tigris.org/issues/show_bug.cgi?id=4001 (update
fails to replace a file external with an actual node)

It's about replacing a file external with a "normal" versioned node,
in one update. But your description might be some other manifestation
of the same problem ... not sure.

> How can I get svn to fetch me the file referenced in svn:externals 
without
> performing an "svn update --depth infinity"?
>
> Remind me again why externals are only processed when depth is infinity?

I suppose this issue is the "reason":

  http://subversion.tigris.org/issues/show_bug.cgi?id=3311 (externals
are not created unless depth=infinity)

But I'm not sure (perhaps that issue is only about dir externals).

-- 
Johan




Re: Error processing externals E205011 and W155035 with simple update

2013-08-10 Thread dlellis
Hi Johan,

I'll try and use the reproduction template, but will need to work this on 
Monday as it appears I'll need to work the template a bit to work with 
http://. 

Here's the quick recipe

Create a file external in svn:externals and commit it and perform full 
update  (foo.c rev 1).
Create a new revision of the file and commit (foo.c rev 2)
Edit the svn:externals to point to latest revision (foo.c rev 2)

Now here's the magic to reproduce the issue:
;:
Only perform an "svn update . -depth empty --ignore-externals" // this 
should bring in the new svn:externals property with foo.c rev 2
Only update the file as "svn update foo.c"

This should now show the file as being switched with an "svn status -v 
foo.c":

Performing a full "svn update" will cause the errors and seem to put the 
WC into a bad state.

A full output of the command prompt is in the listing below.

I appreciate the help!

Dan




From:   Johan Corveleyn 
To: dlel...@rockwellcollins.com
Cc: "users@subversion.apache.org" 
Date:   08/10/2013 03:40 PM
Subject:Re: Error processing externals E205011 and W155035 with 
simple update



Like I replied to your other post: please give us some more concrete
information about this "known slowness of svn update"?

That said, if you can corrupt your working copy by using a series of
regular svn commands, that's clearly a bug.

Would it be possible for you to write down a reproduction recipe
starting from a clean repository? Or even better: a reproduction
script, for example starting from this template:

  http://subversion.apache.org/docs/community-guide/repro-template.bat

That would help a lot in understanding and reproducing the issue.

-- 
Johan

On Sat, Aug 10, 2013 at 3:58 AM,   wrote:
> Hello everyone,
>
> I have confirmed that this behavior also exists in SVN 1.7.5 on Win7.  I
> appreciate any thoughts any one might have.
>
> Thanks,
> Dan
>
>
>
>
> From:dlel...@rockwellcollins.com
> To:users@subversion.apache.org
> Date:08/09/2013 03:55 PM
> Subject:Error processing externals E205011 and W155035 with 
simple
> update
> 
>
>
>
> Hello,
>
> I am running SVN 1.8.1 under Win7.   We have an extensive collection of 
file
> externals that we use to manage shared code between projects (hundreds 
of
> files per working directory).
>
> As known, the performance when performing an SVN Update is slow.  To 
work
> around this, we're attempting to only update externals that are out of 
date
> (by comparing the local revisions of each file to what is in the
> svn:externals.  As we iterate across the files, if we see an external 
out of
> date, we execute an svn update .
>
> Doing this, I'm starting to get corrupted externals  I can reproduce
> this using the following the steps listed below.  Even if I am 
performing
> operations out of order with what is expected, I'd expect that I'd be 
unable
> to harm or corrupt the working directory
>
>
> I basically create a clean checkout of a repository.
> Verify everything is cool, but realize one file external needs a new
> revision.
> I edit the property (using TSVN) and commit the property.
> I then update the working directory ignoring externals and only for the
> immediate directory (because external updates are slow).
> I then update the file I know is out of date (because I just edited the
> svn;externals).
> Ouch, I get an error and it reports that the file has been switched.
> The only way I can repair this is to delete the checkout and recreate 
it.
>
> I believe this also occurs under 1.7, but will have to confirm later.
>
> Can any confirm first that this is indeed an issue, and second how to
> resolve this?
>
> Thanks!
> Dan
>
>
> c:\Project_files\Test>svn status -v --depth immediates
>
>   2850 2850 dlellis  .
>
> X 2818 2818 dlellis  1.txt
>
> X 2633 2633 dlellis  6.txt
>
> X 2626 2626 dlellis  7.txt
>
>   2850 2847 dlellis  CLP
>
>   2850 2783 dlellis  SM
>
> X 2812 2812 dlellis  checkintest.txt
>
> X 2814 2814 dlellis  commit.vbs
>
> X 2728 2728 dlellis  project.conf
>
> c:\Project_files\Test>svn pg svn:externals
>
> http://test/pd_sandbox/plcm/swdb/M4000/project.conf@2728 project.conf
>
> http://test/pd_sandbox/plcm/swdb/Temp/6.txt@2633 6.txt
>
> http://test/pd_sandbox/plcm/swdb/Temp/7.txt@2626 7.txt
>
> http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2818 1.txt
>
> http://test/pd_sandbox/plcm/swdb/Temp/

Re: Error processing externals E205011 and W155035 with simple update

2013-08-11 Thread dlellis
Johan Corveleyn  wrote on 08/11/2013 03:09:55 AM:

> From: Johan Corveleyn 
> To: dlel...@rockwellcollins.com
> Cc: "users@subversion.apache.org" 
> Date: 08/11/2013 03:10 AM
> Subject: Re: Error processing externals E205011 and W155035 with simple 
update
> 
> [ In general, this list prefers bottom-posting (i.e. putting your
> reply at the bottom) or inline replying. More below ... ]

Roger that!

> 
> On Sun, Aug 11, 2013 at 12:56 AM,   wrote:
> > Hi Johan,
> >
> > I'll try and use the reproduction template, but will need to work this 
on
> > Monday as it appears I'll need to work the template a bit to work with
> > http://.
> 
> I don't think the bug is related to the protocol. From what I've read
> so far, I think you should be able to reproduce this with file:// just
> as well (if not: that would be interesting additional information).
> 
> Note that you can use "repository-relative externals" with ^/ etc, so
> you don't have to specify the protocol in your externals definition.

Sorry, I wasn't implying that the bug is HTTP related, just that our 
enterprise only allows http access to our SVN repos.  That said, I'm also 
not sure that I can run the template as it appears to require creating a 
clean repo and we, as lowly developers, are not trusted with such power.

> 
> > Here's the quick recipe
> >
> > Create a file external in svn:externals and commit it and perform full
> > update  (foo.c rev 1).
> 
> Please give the exact value you put into the property. There are a
> couple of variants of the syntax, and it's not clear whether you use a
> peg revision etc.

This is what SVN reports from an svn pg svn:externals before and after the 
update

http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2818 1.txt
http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2827 1.txt

This issue appears if I either use TSVN to enter the above to enter the 
local path, URL, and peg revision OR if I programatically perform this 
through command line SVN as in :
svn propset svn:externals  -F "tmp81A7.tmp" "C:\Project_files\test" where 
the contents of the tmp file are:

http://test/pd_sandbox/plcm/swdb/Test/project.conf@2728 project.conf
http://test/pd_sandbox/plcm/swdb/Temp/6.txt@2633 6.txt
http://test/pd_sandbox/plcm/swdb/Temp/7.txt@2626 7.txt
http://test/pd_sandbox/plcm/swdb/Temp/1.txt@2827 1.txt
http://test/pd_sandbox/plcm/swdb/Temp/commit.vbs@2814 commit.vbs
http://test/pd_sandbox/plcm/swdb/Temp/checkintest.txt@2812 checkintest.txt

What is really interesting (to me at least), is that SVN appears to think 
the file is then switched (shows S with an svn info 1.txt)

 c:\Project_files\Test>svn status -v --depth immediates

   2851 2851 dlellis  .
 S 2851 2827 dlellis  1.txt


> 
> > Create a new revision of the file and commit (foo.c rev 2)
> > Edit the svn:externals to point to latest revision (foo.c rev 2)
> 
> And the exact syntax of the new value.
> 
> >
> > Now here's the magic to reproduce the issue:
> > ;:
> > Only perform an "svn update . -depth empty --ignore-externals" // this
> > should bring in the new svn:externals property with foo.c rev 2
> > Only update the file as "svn update foo.c"
> >
> > This should now show the file as being switched with an "svn status -v
> > foo.c":
> >
> > Performing a full "svn update" will cause the errors and seem to put 
the WC
> > into a bad state.
> 
> 
> -- 
> Johan

Thanks,
Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-11 Thread dlellis
Johan Corveleyn  wrote on 08/11/2013 03:24:31 AM:

> > Hi Johan,
> >
> > I think you are spot on on both issues - thank you.
> 
> You're welcome :-).
> 
> > The performance issue relates to SVN externals stem, in my case, from 
new
> > connections for each http request for externals versus keeping an open 
one
> > for non-externals.  Each connection results in a second or so delay.
> > Multiply this by hundreds and updates that would take seconds, 
nowtakes ten
> > minutes.  There's been quite a bit of discussion about pooling 
connections I
> > believe, but I don't know the current state of this effort or if its 
in any
> > plan.
> 
> Hm, there were indeed some discussions (and patch attempts) recently,
> but I'm not sure of the current state either. There is an open issue
> about this in the issue tracker:
> 
>   http://subversion.tigris.org/issues/show_bug.cgi?id=3763 (checkout
> should re-use ra session for externals if possible)
> 
> It's marked as "started", but I'm not sure if that's correct status at
> this point.
> 
> I'm cc'ing Ben Reser (who has the above issue assigned to him) and
> Ivan Zhakov (who recenty wrote a patch for this). Perhaps they can
> shed some light on the current state of "RA session caching".

Thank you.

> 
> As far as I remember from discussions during the Berlin Hackathon,
> this was deemed too risky for 1.8, but perhaps within scope for 1.9.
> 
> >
> > All that said, is there a work around to convincing SVN to recreate 
the file
> > without a full fledge update?
> 
> No, I'm afraid I know of no workaround.
> 
> Perhaps some further variation of what you're already trying (but I'm
> sure you did already many trial and error attempts).

I think a short term workaround would be to detect if an external file 
name path was changed and force a full update.  For revision updates only, 
I believe I could still make this solution work (update local properties 
and iterate over externals that do not match, updating one by one).  This 
will give us a huge performance boost for some of our cases.

Thanks for the help!

Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-11 Thread dlellis
Ben Reser  wrote on 08/11/2013 09:54:43 AM:

> 
> On Sun, Aug 11, 2013 at 3:24 AM, Johan Corveleyn  
wrote:
> > I'm cc'ing Ben Reser (who has the above issue assigned to him) and
> > Ivan Zhakov (who recenty wrote a patch for this). Perhaps they can
> > shed some light on the current state of "RA session caching".
> >
> > As far as I remember from discussions during the Berlin Hackathon,
> > this was deemed too risky for 1.8, but perhaps within scope for 1.9.
> 
> It's on my todo to review Ivan's patch.  It seems like the best short
> term solution and is basically what I had already intended to do.

I'd be willing to provide some real world testing on this patch.  I'm not 
sure I'd be able to get up to speed sufficiently on test models and the 
like, but would do what I can.  This would be a huge boon for us as I know 
our place has two major toolsets in place that make extensive use of 
externals and the session caching (especially for satellite locations) 
performance issue is the number one frustration with the toolsets.

Thanks,
Dan


Re: Error processing externals E205011 and W155035 with simple update

2013-08-11 Thread dlellis
Johan Corveleyn  wrote on 08/11/2013 02:44:38 PM:

> >> > Here's the quick recipe
> >> >
> >> > Create a file external in svn:externals and commit it and perform 
full
> >> > update  (foo.c rev 1).
> >>
> 
> Okay, with this information I can reproduce the issue (both with 1.7.9
> and with trunk@1512490):
> 
> 
> I get it also when only using --ignore-externals (or --depth=empty),
> i.e. it's not necessary to combine both options in the step before the
> final update.
> 
> Dan, can you please file an issue for this (referencing this thread)?
> 

Thank you for reproducing this!  I thought I was going crazy for a 
while...  I've entered a new issue per your recommendations.

http://subversion.tigris.org/issues/show_bug.cgi?id=4409


Thanks,
Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-14 Thread dlellis
Ben Reser  wrote on 08/11/2013 09:54:43 AM:

> From: Ben Reser 
> To: Johan Corveleyn 
> Cc: dlel...@rockwellcollins.com, "users@subversion.apache.org" 
> , Ivan Zhakov 
> Date: 08/11/2013 09:55 AM
> Subject: Re: How to change paths on an external file without a full 
> update --depth infinity?
> 
> On Sun, Aug 11, 2013 at 3:24 AM, Johan Corveleyn  
wrote:
> > I'm cc'ing Ben Reser (who has the above issue assigned to him) and
> > Ivan Zhakov (who recenty wrote a patch for this). Perhaps they can
> > shed some light on the current state of "RA session caching".
> >
> > As far as I remember from discussions during the Berlin Hackathon,
> > this was deemed too risky for 1.8, but perhaps within scope for 1.9.
> 
> It's on my todo to review Ivan's patch.  It seems like the best short
> term solution and is basically what I had already intended to do.

Hi,

Will this look like a 1.8.x change in your mind?  We are really sitting on 
pins and needles with excitement for this!

I like Subversion - simple, powerful, and light weight. 

I've been a part of many trade studies over the years looking at how 
different groups choose the config mgmt tool of their choice (CC, CMVC, 
PVCS, Git, SVN, etc. have all been used).  When it comes to product line 
strategy, the trade studies have typically given low marks to Subversion 
based on external performance (and certainly other open tools for other 
reasons too). 

Typically the results of these trade studies never get flowed back out as 
to why a tool wasn't chosen.  I'm now seeing that this is unfortunate for 
open tools as they may not always see the desires of certain industries. 
In this case, as a new person to the SVN community, I admit I was 
surprised to see that the chatter within the SVN community didn't match 
the chatter I've been hearing for years among the various organizations 
within my company (we're fairly large ~20k ppl and of which the different 
organizations are largely independent and focused on vary different 
products yet we came to the same conclusions - externals are invaluable, 
yet painful). 

I believe that if we can improve external performance (speed and 
integration -- like handling externals when depth != infinity), not only 
would we help the current users of SVN that have come to accept this, but 
we would have a huge opportunity to get back on the radar of other users 
that have previously chosen other options.

I completely appreciate the work you all do, SVN is a great tool.  This is 
just some feedback from a user group that I'm concerned you may not have 
from properly.

Going forward, I think a takeaway for me is that as industry uses open 
source tools and subsequently performs trade studies on those tools, its 
important to provide feedback to the open source community.  Its not 
especially fair to use and not contribute in some form.

I'm willing to help perform informal testing or in other tasks that could 
be useful.

Thanks,
Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-14 Thread dlellis
Les Mikesell  wrote on 08/14/2013 10:17:41 AM:

> From: Les Mikesell 
> To: dlel...@rockwellcollins.com
> Cc: Ben Reser , Ivan Zhakov ,
> Johan Corveleyn , "users@subversion.apache.org" 
> 
> Date: 08/14/2013 10:18 AM
> Subject: Re: How to change paths on an external file without a full 
> update --depth infinity?
> 
> On Wed, Aug 14, 2013 at 11:48 AM,   wrote:
> >
> > I believe that if we can improve external performance (speed and 
integration
> > -- like handling externals when depth != infinity), not only would we 
help
> > the current users of SVN that have come to accept this, but we would 
have a
> > huge opportunity to get back on the radar of other users that have
> > previously chosen other options.
> 
> I'm not sure that current SVN users accept problems with depth !=
> infinity as much as they arrange their layout so they don't have to do
> that.   What's a common use case for needing some disjoint arrangement
> of components that you can't assemble with externals and normal
> recursion?
> 

This is a case of trying to improve performance on externals by only 
updating externals that have changed.  Without connection caching, 
performing an external update over a WAN is a test of patience.  For us, 
our repo is accessed over a WAN.  Its not an issue for non-"file 
externals".  For example, a WC of 1000 file externals will take over 15 
minutes to update with zero changes but the same WC with no file externals 
(1000 normal files) takes 30 seconds tops.  Keep in mind, no actual file 
revisions are being downloaded, just checked.

We have been attempting to put in place a workaround to give us the 
benefit of file externals with a general algorithm of:

update external table (svn ps svn:external -F tmpfile)
commit external table (svn commit . --depth empty --ignore_externals)
update the externals table for paranoia (svn update . --depth empty 
--ignore_externals)
foreach external in svn:properties (compare svn:externals to svn info -v)
  if the local revision (svn info) is not the same as the external
 update the external (svn update foo.c)

This has proven to be difficult to do since:
a. a corruption issue was identified (bug 4409)
b. we can't change a path on a file external (bug 4001) with out a full 
update (depth infinity) (old file gets deleted, new one doesn't get 
brought in)

Either of these issues prevents doing a simple update of a file external 
or doing a current directory update (50 files is more bearable than a full 
1000).

That said, this attempted work around would not be needed if the 
connection caching can bring file external updates to something closer to 
the performance of regular files. 

Hope this helps, I'd be glad to explain in more detail how we implement 
this.  I think we have a very successful strategy for our needs.

Thanks
Dan

 




Re: How to change paths on an external file without a full update --depth infinity?

2013-08-14 Thread dlellis
Kevin M Radke/CedarRapids/RockwellCollins wrote on 08/14/2013 11:52:44 AM:


> dlel...@rockwellcollins.com wrote on 08/14/2013 01:13:52 PM:
> > > I'm not sure that current SVN users accept problems with depth !=
> > > infinity as much as they arrange their layout so they don't have to 
do
> > > that.   What's a common use case for needing some disjoint 
arrangement
> > > of components that you can't assemble with externals and normal
> > > recursion?
> > > 
> > 
> > This is a case of trying to improve performance on externals by only
> > updating externals that have changed.  Without connection caching, 
> > performing an external update over a WAN is a test of patience.  For
> > us, our repo is accessed over a WAN.  Its not an issue for non-"file
> > externals".  For example, a WC of 1000 file externals will take over
> > 15 minutes to update with zero changes but the same WC with no file 
> > externals (1000 normal files) takes 30 seconds tops.  Keep in mind, 
> > no actual file revisions are being downloaded, just checked. 
> > 

> 
> Even with connection caching, you will still be asking the 
> server a separate question for each external instead of one 
> question for the whole working copy.  I think there will 
> always be a performance cost to using externals, and using 
> thousands is just asking for problems. 

We're ok with _some_ performance cost.  Also, it might be valuable to the 
SVN community to understand other use cases when they make architectural 
decisions.  I'm not sure its technically impossible to combine all the 
external requests by directory into a single transaction.  Let's ask the 
community and developers what they think.

> Designing a build 
> management system on top of subversion using only externals 
> is risky.

I disagree, but we've had this conversation already.  I'd be very welcome 
to try anything to help our performance.


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-14 Thread dlellis
Les Mikesell  wrote on 08/14/2013 01:58:29 PM:

> From: Les Mikesell 
> To: dlel...@rockwellcollins.com
> Cc: kmra...@rockwellcollins.com, Ben Reser , Ivan
> Zhakov , Johan Corveleyn , 
> "users@subversion.apache.org" 
> Date: 08/14/2013 01:58 PM
> Subject: Re: How to change paths on an external file without a full 
> update --depth infinity?
> 
> On Wed, Aug 14, 2013 at 2:04 PM,   wrote:
> >
> >
> >> Designing a build
> >> management system on top of subversion using only externals
> >> is risky.
> >
> > I disagree, but we've had this conversation already.  I'd be very 
welcome to
> > try anything to help our performance.
> 
> Externals can be very useful to assemble mixed revisions of
> separately-versioned components, but why do they have to be
> file-level?  Are there other tools involved in the process that do not
> understand subdirectories or symlinks?  Or just no natural groupings?
> 

No natural groupings.  In embedded, safety critical DO-178b products, we 
have very rigorous rules that essentially prevent one size fits all 
developments.  Product line management on things like this become very 
hard and tend to create clone and owns of the code base as we move from 
one system to another.  Requirements and verification (including 
structural coverage testing) is a beast.

We did a study of similar projects and found that in unexpected amount of 
code is common that one wouldn't expect to be common.  This ad hoc common 
code was larger than the amount we could actually purposely create as 
"designed for reuse" (libraries or directory level commonality).  I'd 
imagine if anyone took a baseline and forked it, did a bunch of work and 
compared it back to the baseline, you'd be surprised what turns out as 
still being common (unless someone does a ton of style changes).  Now try 
and imagine how to extract those random bits of code into groupings.  It 
can be done, but in our world, you can't go off and willy nilly change and 
refactor code.  Its nice to maintain a living pedigree with your baseline. 
 

Now, in our case, we do stuff for aircraft,... wouldn't it be nice to 
maintain living pedigrees with all similar models of aircraft?  Fix an 
issue in one place and advertise it to all the others.  File externals 
give you this.  It fits very well into the embedded safety critical world 
in the "don't touch that code unless you have to" and "let's fix it once". 
 Refactoring code in this world just can't happen as often as you'd like 
(its also a chance to reinject bugs).

Hope this helps!


Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-14 Thread dlellis
Les Mikesell  wrote on 08/14/2013 02:49:38 PM:


> On Wed, Aug 14, 2013 at 4:23 PM,   wrote:
> >
> > Now, in our case, we do stuff for aircraft,... wouldn't it be nice to
> > maintain living pedigrees with all similar models of aircraft?  Fix an 
issue
> > in one place and advertise it to all the others.  File externals give 
you
> > this.  It fits very well into the embedded safety critical world in 
the
> > "don't touch that code unless you have to" and "let's fix it once".
> > Refactoring code in this world just can't happen as often as you'dlike 
(its
> > also a chance to reinject bugs).
> >
> > Hope this helps!
> 
> So the point is to intentionally pull the HEAD revision of a whole
> bunch of files together where each is located arbitrarily  and can
> change independently?   I guess that's about the opposite of the way I
> think of version control, so I can't suggest anything else.   Are
> there enough different 'top level' collections or a fast enough change
> that you can't simply copy the right files into place instead of
> having external references there?

No HEAD revisions, all files are pegged. 

For example, if everyone is linked to a common file (foo.c, revision 20, 
project A "pedigree") and a bug is fixed, each project will see that a fix 
has been made.  Each project has to make a decision: to remain on the 
current revision (no budget or schedule to update), update to the latest 
(if they have budget and schedule), or to "fork" (if they don't like the 
fix and have to implement a new feature).  This can happen any time, not 
necessarily when the file is committed.

Each new project inherits the externals from their baseline (they do not 
create new links or go out and search for reuse - that'd be a pain!)  So 
until a project decides to "fork" a file, they see (but not automatically 
receive) all the changes made to that file.  Nothing happens to your 
project without explicit action. 

Each file lives in a special area of the repository that identifies its 
pedigree.  Code your project writes automatically becomes usable by any 
other project.

This is the same approach Rational CMVC takes.

Hope this helps,
Dan




Re: How to change paths on an external file without a full update --depth infinity?

2013-08-14 Thread dlellis
> >
> > No HEAD revisions, all files are pegged.
> >
> > For example, if everyone is linked to a common file (foo.c, revision 
20,
> > project A "pedigree") and a bug is fixed, each project will see that a 
fix
> > has been made.  Each project has to make a decision: to remain on the
> > current revision (no budget or schedule to update), update to the 
latest (if
> > they have budget and schedule), or to "fork" (if they don't like the 
fix and
> > have to implement a new feature).  This can happen any time, not 
necessarily
> > when the file is committed.
> 
> Then what makes the collection of pegged externals easier to manage
> than explicitly copying specific revisions into the top level where
> the reference makes it appear and subsequently merging changes or
> deleting files and copying in the updated version.  Those operations
> are relatively efficient with svn and copies carry along the revision
> history.
> 
> I don't follow how each project 'sees' that fixes have been made - you
> shouldn't see that through a pegged external.

We have a tool that mimics explorer.  It queries the repo for the latest 
revision on each file external (yes a bit talky with the server).  If they 
are not identical, it displays revision x of y.  It also displays the 
pedigree (or "history"" in CMVC lingo).  You can select each file, and 
graphically decide to "fork" to a different pedigree/history, or peg to a 
different revision.

> > Each new project inherits the externals from their baseline (they do 
not
> > create new links or go out and search for reuse - that'd be a pain!) 
So
> > until a project decides to "fork" a file, they see (but not 
automatically
> > receive) all the changes made to that file.  Nothing happens to your 
project
> > without explicit action.
> 
> You'd get roughly the same effect with an 'svn cp' of a baseline - the
> equivalent of a branch or tag even if you name it something else.

I don't think this will do what we want.

> 
> > Each file lives in a special area of the repository that identifies 
its
> > pedigree.  Code your project writes automatically becomes usable by 
any
> > other project.
> 
> None of that would need to change regardless of whether you copy a
> revision into a different folder of reference it directly with an
> external.   As long as you aren't floating to the HEAD version you
> have to do something to bump the revisions - why not just copy it in
> the repo where remote revision checks will be fast?

Once you copy, you break the link.  If you were to make a change to the 
copy, no one else would then see it. 

You are correct, you could replicate all of this many different ways. When 
we did our study, externals (aside from the performance issue) was the 
best answer.

I wish I could give a quick tour of the process/tool.  It'd answer so many 
questions.

Thanks,
Dan



Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
> > Once you copy, you break the link.  If you were to make a change to 
the 
> > copy, no one else would then see it. 
> 
> No one else would see it with externals either, except that you 
> wrote a custom tool to analyze the externals, see if a newer 
> revision of the original exists, and show that to the user. If you 
> can do that with externals, you can do that with copies too. (Use 
> "svn log --stop-on-copy" to find out where the copy came from, then 
> see if there are newer revisions of that.)

The challenge I then see on this is one of finding all instances of foo.c. 
 If you have foo.c copied/forked fifty times to different projects, each 
of which has branched a couple of times, how do you programmatically find 
all different instances of foo.c (to let a developer choose which may be 
most appropriate?)  If you have good ideas, I'm very open to listening.

Also if you have to projects that both want foo.c and both have valid 
changes to make to the file, how does that get managed when they are 
copies?  Its a trivial implementation when it is implemented as a file 
external. 

We also have instances where we purposely want multiple copies of the same 
exact file within the same project.  We can effectively manage this 
through file externals to a structured "datastore" (AKA a set of folders 
within a repo).  Regardless of where and how a team decides to structure 
their project, all files are neatly organized in this one section of the 
repo (that is considered taboo to directly interact with).  The ability to 
have a specific file having many "copies" of itself and not care about its 
position within the repository is a powerful feature.  I understand this 
may diverge a bit from SVN's core thoughts on CM, but if SVN can support 
odd variations to its use, it becomes an even more indispensable building 
block.  Diversity in approaches is good.

>From a feature perspective, externals are a very appropriate method to 
accomplish this (really a CM implementation of symlinks).  If we're saying 
that externals from an implementation standpoint are not quite appropriate 
at this time, I get that argument.  What is the general consensus as to 
where externals are on the roadmap? 

I may not convince the team that externals are really really useful (even 
if abused) in this application, but I'm hoping that the team does 
appreciate the general usefulness of externals and keeps maturing the 
feature.

Thanks
Dan



Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
> > The challenge I then see on this is one of finding all instances of 
foo.c.
> > If you have foo.c copied/forked fifty times to different projects, 
each of
> > which has branched a couple of times, how do you programmatically find 
all
> > different instances of foo.c (to let a developer choose which may be 
most
> > appropriate?)  If you have good ideas, I'm very open to listening.
> 
> There is no difference in that question than finding where the
> 'future' copies of a pegged external target went.  You can only do
> either if you have a convention for a canonical path.

true (I believe).

> 
> > Also if you have to projects that both want foo.c and both have valid
> > changes to make to the file, how does that get managed when they are 
copies?
> > Its a trivial implementation when it is implemented as a file 
external.
> 
> How so?  I assume you also have to handle cases either way: where both
> projects want the same change and where both projects need different
> changes - where typical svn users would have branches/tags to
> distinguish them.   But regardless of how you identify the target
> file, there shouldn't be any effective difference between copying a
> version into your directory or using a file external as long as you
> don't modify it in place and commit it back - something your external
> tool could track.

We do want to modify in place.  Copying back creates an additional step 
that is already managed quite well by SVN with externals.  I don't want to 
duplicate something that already exists (I'll mess it up).  There was some 
discussion on another thread about advancing a peg revision of an external 
when an external is committed.  This would be a neat feature, though I 
completely understand why it would not be incorporated.  We do this behind 
the scenes right now with very little work (post-commit script in TSVN 
gives us knowledge of what a user committed).

> 
> > We also have instances where we purposely want multiple copies of the 
same
> > exact file within the same project.  We can effectively manage this 
through
> > file externals to a structured "datastore" (AKA a set of folders 
within a
> > repo).  Regardless of where and how a team decides to structure their
> > project, all files are neatly organized in this one section of the 
repo
> > (that is considered taboo to directly interact with).  The abilityto 
have a
> > specific file having many "copies" of itself and not care about its 
position
> > within the repository is a powerful feature.  I understand this may 
diverge
> > a bit from SVN's core thoughts on CM, but if SVN can support odd 
variations
> > to its use, it becomes an even more indispensable building block. 
Diversity
> > in approaches is good.
> 
> Again, you get the history in a copy.  You can tell if they are the
> same.  Or, on unix-like systems you can use symlinks to a canonical
> copy within the project.

We're not a unix-like system but that is what would work great (with the 
exception that you can't revision control symlinks, right?)

> 
> > From a feature perspective, externals are a very appropriate method to
> > accomplish this (really a CM implementation of symlinks).  If we're 
saying
> > that externals from an implementation standpoint are not quite 
appropriate
> > at this time, I get that argument.  What is the general consensus as 
to
> > where externals are on the roadmap?
> 
> I agree that externals are very useful, but most projects would use
> them at subdirectory levels for component libraries where they work
> nicely, not for thousands of individual file targets.   Is there
> really no natural grouping - perhaps even of sets of combinations that
> have been tested together that you could usefully group in
> release-tagged directories?

The whole discovery we found is that most of our reuse occurred in 
unplanned ways (I'd imagine if you took two linux distros and compare 
files which changed and didn't change, it would be a huge collection of 
random files that aren't easily abstracted out.  You might be able to do 
it once, but as each new distribution branches out, the commonality 
between each of them becomes impossible to form groupings on.


> ...'just work' without special considerations.  I'm not against better
> performance, of course, but it makes sense to me to make pragmatic
> design decisions for the same reasons you might avoid throwing
> millions of files in one flat directory even in a non-versioned
> scenario.   Theoretically, you should be able to do that, but in
> practice it isn't going to perform as well as something with better
> structure.

I'm not sure what a reasonable number of external files per folder is, but 
I'd think it'd be similar to a reasonable number of regular files would 
be.  Two million is nuts, but 50 seems reasonable.  The issue is that I'm 
currently forced to deal with not just the current directory, but the 
recursion on all nested directories (--depth infinity).  If, as the 
subject of

Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
> On Thu, Aug 15, 2013 at 12:39 PM,   wrote:
> >
> >>  But regardless of how you identify the target
> >> file, there shouldn't be any effective difference between copying a
> >> version into your directory or using a file external as long as you
> >> don't modify it in place and commit it back - something your external
> >> tool could track.
> >
> > We do want to modify in place.  Copying back creates an additionalstep 
that
> > is already managed quite well by SVN with externals.
> 
> I've never done that with a file external - where does the commit go?

It commits a new revision of what the file external pointed to - pretty 
handy.  If you are pegged, it will not automagically update your pegged 
revision (as I'd expect), so unless you are on the HEAD or update your peg 
to what just committed, an update will revert your WC back to the pegged 
version. 

> 
> >> Again, you get the history in a copy.  You can tell if they are the
> >> same.  Or, on unix-like systems you can use symlinks to a canonical
> >> copy within the project.
> >
> >
> > We're not a unix-like system but that is what would work great (with 
the
> > exception that you can't revision control symlinks, right?)
> 
> I think so, but the links and the target would be versioned
> independently which might complicate your tracking.

Yes, it would complicate things quite a bit and introduce areas for 
defects to get introduced.

> >
> > The whole discovery we found is that most of our reuse occurred 
inunplanned
> > ways (I'd imagine if you took two linux distros and compare files 
which
> > changed and didn't change, it would be a huge collection of random 
files
> > that aren't easily abstracted out.  You might be able to do it once, 
but as
> > each new distribution branches out, the commonality between each of 
them
> > becomes impossible to form groupings on.
> 
> I was thinking of just adding an extra layer of grouping management
> that would be versioned and able to be duplicated as much as
> necessary.   Suppose you made 10 directories and copied 100 files into
> each with tagged versions of these directories for every combination
> you need to access.   Normally there would be natural groupings where
> there is a common manager making decisions, etc., but for the moment
> just consider it for performance.   Within the repository, the copies
> are cheap like symlinks - you could have a large number of
> pre-arranged tagged choices.   Then your top level project becomes 10
> directory-level externals instead of 1000 file externals.

With more complexity comes more bugs and process missteps.  We're really 
striving to keep things as simple as possible.  We're fundamentally 
accepting of update times going from 2 seconds to 2 minutes.  Its harder 
when 2 minutes becomes 20 minutes.

> 
> > I'm not sure what a reasonable number of external files per folder is, 
but
> > I'd think it'd be similar to a reasonable number of regular files 
would be.
> > Two million is nuts, but 50 seems reasonable.
> 
> Think of this in terms of client-server activity.  With directory
> level externals, the client can ask the server if anything under the
> directory has newer revisions in one exchange and if it hasn't, you
> are done.   So what's reasonable is the amount of activity you want to
> wait for.

The whole discussion has centered on an attempted work around for the 
connection caching that doesn't currently occur for externals.  If that 
can happen, I think we'd be very content.  We're accepting of some 
performance issues.  There was an XKCD a while ago that talked about how 
much time a task takes, how many times you do it and how much waste is 
created over a year.  It was interesting (even if obvious if you thought 
about it).  I think with connection caching we'd hit the sweet spot and 
working further would result in diminishing returns.  This thread is an 
attempt a hopefully short-term work around this limitation.

> Usually I'd consider the 'human' side of organization first, so if you
> can come up with any groupings that could be done as copies into
> tagged directories you might want to arrange them by the people/groups
> who make the choices - and then the performance win would just be an
> extra bonus.

That's just it, we can't find (let alone maintain over time) any 
consistent groupings by function.  Trying to create groupings other ways 
could confuse the developers, or if we try and hide the fact that we have 
an optimized backend of sorts, we then have to write more tool software 
(we don't like writing tools, we like writing embedded software).  In the 
end, revision controlled symlinks are the best answer and file externals 
appear to be very close.  And we're oh so close with file externals right 
now. 

Thanks
Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
> On Thu, Aug 15, 2013 at 2:03 PM,   wrote:
> >
> >> With more complexity comes more bugs and process missteps.  We're 
really
> > striving to keep things as simple as possible.  We're fundamentally
> > accepting of update times going from 2 seconds to 2 minutes.  Its 
harder
> > when 2 minutes becomes 20 minutes.
> 
> Are your build systems on the other side of the world from the
> repository?   The quick fix might be to reduce the latency one way or
> another (move one of the pieces, use the wandisco distributed repo
> approach, etc?) or automate with something like jenkins so you don't
> have a person waiting.

Yes, and that's a big contributor.  Co-location helps significantly, but 
isn't an option in our case.  I'll take a look at your suggestions.

Thanks
dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
> >
> > It commits a new revision of what the file external pointed to - 
pretty
> > handy.  If you are pegged, it will not automagically update your 
pegged
> > revision (as I'd expect), so unless you are on the HEAD or update your 
peg
> > to what just committed, an update will revert your WC back to the 
pegged
> > version.
> 
> I'd actually expect it to be pretty confusing if you had multiple
> people committing changes based from different back-rev pegged
> references anywhere near the same time frame.  Does your external
> 'notify about new versions' tool help with that?   Don't you get
> conflicting changes that you'd want branches to help isolate?

Yes, its obvious to users when they are not on the latest revision of the 
file, but they c/would still go through a merge process if need be.

Its actually very straight-forward as we intentionally focused on 
targeting non-CM-guru folks.


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
Ben Reser  wrote on 08/15/2013 02:57:23 PM:


> On 8/15/13 1:05 PM, Les Mikesell wrote:
> > I'd actually expect it to be pretty confusing if you had multiple
> > people committing changes based from different back-rev pegged
> > references anywhere near the same time frame.  Does your external
> > 'notify about new versions' tool help with that?   Don't you get
> > conflicting changes that you'd want branches to help isolate?
> 
> The commit won't complete you'll get an out of date error.

That's right, isn't it.  It'd be no different than two folks trying to 
commit the same file around the same time, right (one would get an out of 
date error)? 

Thanks,
Dan


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-15 Thread dlellis
> >> The commit won't complete you'll get an out of date error.
> >
> > That's right, isn't it.  It'd be no different than two folks trying to
> > commit the same file around the same time, right (one would get an out 
of
> > date error)?
> 
> Right, but when working on the trunk explicitly you'd expect to update
> to accept others' changes often, or to branch to preserve and isolate
> your differences.   I don't see how either quite matches a model where
> changes might be made based on multiple differing back-rev pinned
> externals.   What happens if two projects don't want to accept each
> others' changes and need to commit their own?  In a more typical
> scenario they would be working on branch copies that could diverge,
> but I think you lose that by forcing a canonical path for development
> (as a tradeoff for knowing where the 'new' work is...).
> 
If a project doesn't want to accept a change, they "fork" to a new 
"history".  The tool does this with a svn cp  
 and an update to the svn:externals property.  They 
now lose sight of what the other project commits after that fork though. 
The backend of where the file is stored and how is masked by the tool. 
 is essentially implemented as a folder.  To the developer, they 
may know that their file is really a file external, but they don't treat 
it really any different from a normal file until it comes time to "fork". 
I try to differentiate "forking" as a pedigree/history from branching and 
the like. 

This system is essentially an implementation of Rational's CMVC history 
feature.


Re: How to change paths on an external file without a full update --depth infinity?

2013-08-16 Thread dlellis
> >
> > If a project doesn't want to accept a change, they "fork" to a new
> > "history".  The tool does this with a svn cp 
> >  and an update to the svn:externals property. They 
now
> > lose sight of what the other project commits after that fork though. 
The
> > backend of where the file is stored and how is masked by the tool.
> >  is essentially implemented as a folder.  To the developer, 
they
> > may know that their file is really a file external, but they don't 
treat it
> > really any different from a normal file until it comes time to "fork". 
 I
> > try to differentiate "forking" as a pedigree/history from branching 
and the
> > like.
> >
> > This system is essentially an implementation of Rational's CMVC 
history
> > feature.
> 
> In subversion's view a copy is a branch so any distinction is strictly
> your own convention.  Likewise for tags, except that there is a
> generally accepted convention of not committing changes after a tag
> copy.   Do you have additional conventions or tools to help users of
> the pre-fork version know that it has branched?I don't think there
> is a generic solution for that - subversion tracks copy-from history,
> but not copy-to.

No.  There is no way to know who is using a fork you may have created or 
who has forked from someplace (short of scanning all projects of course). 
I'm not sure fork is the best name to give this concept, but we didn't 
want to choose branch or tag for obvious reasons



svn log for externals

2013-08-29 Thread dlellis
Hello everyone,

I thought I've seen discussion in the past on SVN behavior when doing an 
"svn log" on a WC with externals but my google skills seem weak.

Currently svn log will display file changes and svn:externals property 
changes, but will not display changes made to the file pointed at by the 
external.  For example, if a user commits a change to a file external, svn 
log will show that the properties reflects a new revision of the file is 
now being referenced, but will not see commit made to the file, even if it 
was done from the current WC.

It would be a very useful feature to have svn log include the log of the 
file external(s) as if it were a formal vile within the WC (even if it 
required a switch --include-externals) to enable.

Any thoughts on this behavior?

Thanks,
Dan






Programatically determine revision of commit

2013-10-03 Thread dlellis
Hi,

We're attempting to write some scripts to perform automatic commits for 
certain files under certain conditions.  We wish to use the just committed 
revision number to perform some other logging operation.  The --xml 
command line option on many of the other commands (info, status, etc.) are 
fantastic, but the commit command does not allow this.

I'm currently attempting to parse the standard output of the commit 
message to grab it, but feel this is less than optima as it seems very 
error prone.

Is there any consideration to add an --xml option to the commit command 
output or a better programmatic means to grab the commit revision?

c:\Project_files\sandbox_v2>svn commit 1.txt -m "Checkin message".
Sending1.txt
Transmitting file data .
Committed revision 3272.

c:\Project_files\sandbox_v2>

Thanks,
Dan


Inconsistency with --xml for svn info and svn status ?

2014-01-24 Thread dlellis
Hi,

I think I found a subtle difference for outputting XML formatted data from 
svn info and status.  I'm on Win7, SVN 1.8.5 (r1542147).

svn status when generating  tags does not seem to care about 
your current working directory (CWD) relative to the requested path.  It 
always appears to return an absolute file path.
svn info returns different path depending on if the WC path requested is 
within the EXACT spelling of the CWD.  When your requested path is within 
the enclosing CWD, it will return relative paths and absolute otherwise. 
If the requested path spelling is equivalent (for Win systems any how), it 
will only return relative for exact spelling.  Changing the case of a 
directory will change it to absolute.  It appears there may be other ways 
to get relative vs absolute with info but I couldn't repeat reliably.

It would be nice if SVN would always return an absolute path consistent 
with the status command.  This would also solve the case where Win paths 
are case insensitive and equivalent paths return different results.  It 
would also be good as generally when using --xml, you are using some 
automated tool or application and do not always know or control what the 
CWD is (especially for Win apps).  Additionally, the path that the entry 
is relative to is not the WC but the CWD.  This makes grabbing an XML file 
later and making sense of it impossible (since the CWD is not embedded 
within the XML file) -- see third example below.

I'm not sure if this is considered operating as-design, and therefore an 
enhancement request, or if it is a bug. 

I appreciate your time!

Thanks
Dan


Here's an example... This first call is using a case-different path from 
what the CWD is (note the project_files vs Project_files).  It returned an 
absolute path. The second returned relative paths (exact spelling match). 
The third is showing the relative paths are not relative to a WC, but 
relative to the CWD.

c:\Project_files\LocalSVN>svn info 
"c:\project_files\LocalSVN\trunk\info_test" --depth immediates --xml



(Snipped)


(Snipped)




c:\Project_files\LocalSVN>svn info 
"c:\Project_files\LocalSVN\trunk\info_test" --depth immediates --xml



(Snipped)


(Snipped)



c:\>svn info "c:\Project_files\LocalSVN\trunk\info_test" --depth 
immediates --xml



(Snipped)

(Snipped)