"svn revert" corrupts working copy

2014-02-19 Thread Florin Avram

Hi,

I found a situation when using "svn revert" seems to corrupt the working 
copy. In a working copy like this:

- wc_root_dir
-- directory
--- file

erase the directory from disk (not using SVN) and create a file with the 
same name instead.


"svn status" will report:
~ directory
! directory/file

Now, see the following sequence:

svn revert directory/file
svn: E155009: Failed to run the WC DB work queue associated with 
'/wc_root_dir/directory/file', work item 2 (file-install directory/file. 
1 0 1 1)
svn: E720087: Can't move '/wc_root_dir/.svn/tmp/svn-3884C5DC' to 
'/wc_root_dir/directory/file': The parameter is incorrect.


svn status
svn: E155037: Previous operation has not finished; run 'cleanup' if it 
was interrupted


svn cleanup
Gives same message as "svn revert".

svn --version
svn, version 1.8.5 (r1542147)
   compiled Nov 27 2013, 04:10:18 on x86_64/x86-microsoft-windows5.1.2600

Regards,
Florin Avram


Re: "svn revert" corrupts working copy

2014-02-19 Thread Stefan Sperling
On Wed, Feb 19, 2014 at 12:50:02PM +0200, Florin Avram wrote:
> Hi,
> 
> I found a situation when using "svn revert" seems to corrupt the working
> copy. In a working copy like this:
> - wc_root_dir
> -- directory
> --- file
> 
> erase the directory from disk (not using SVN) and create a file with the
> same name instead.
> 
> "svn status" will report:
> ~ directory
> ! directory/file
> 
> Now, see the following sequence:
> 
> svn revert directory/file
> svn: E155009: Failed to run the WC DB work queue associated with
> '/wc_root_dir/directory/file', work item 2 (file-install directory/file. 1 0
> 1 1)
> svn: E720087: Can't move '/wc_root_dir/.svn/tmp/svn-3884C5DC' to
> '/wc_root_dir/directory/file': The parameter is incorrect.

Subversion is (correctly) trying to put a directory where you put a file.
Until the obstructing file is removed you'll get this error.
This is by design.

> 
> svn status
> svn: E155037: Previous operation has not finished; run 'cleanup' if it was
> interrupted
> 
> svn cleanup
> Gives same message as "svn revert".
> 
> svn --version
> svn, version 1.8.5 (r1542147)
>compiled Nov 27 2013, 04:10:18 on x86_64/x86-microsoft-windows5.1.2600
> 
> Regards,
> Florin Avram


Re: "svn revert" corrupts working copy

2014-02-19 Thread Florin Avram
Well, this design corrupts the working copy and I cannot use it after. I 
doubt this is what you want...
I ran into this issue by testing how "svn revert" behaves on different 
strange situations, but, at least theoretically, anybody can run into it.



On 19.02.2014 13:02, Stefan Sperling wrote:

On Wed, Feb 19, 2014 at 12:50:02PM +0200, Florin Avram wrote:

Hi,

I found a situation when using "svn revert" seems to corrupt the working
copy. In a working copy like this:
- wc_root_dir
-- directory
--- file

erase the directory from disk (not using SVN) and create a file with the
same name instead.

"svn status" will report:
~ directory
! directory/file

Now, see the following sequence:

svn revert directory/file
svn: E155009: Failed to run the WC DB work queue associated with
'/wc_root_dir/directory/file', work item 2 (file-install directory/file. 1 0
1 1)
svn: E720087: Can't move '/wc_root_dir/.svn/tmp/svn-3884C5DC' to
'/wc_root_dir/directory/file': The parameter is incorrect.

Subversion is (correctly) trying to put a directory where you put a file.
Until the obstructing file is removed you'll get this error.
This is by design.


svn status
svn: E155037: Previous operation has not finished; run 'cleanup' if it was
interrupted

svn cleanup
Gives same message as "svn revert".

svn --version
svn, version 1.8.5 (r1542147)
compiled Nov 27 2013, 04:10:18 on x86_64/x86-microsoft-windows5.1.2600

Regards,
Florin Avram




Re: "svn revert" corrupts working copy

2014-02-19 Thread Stefan Sperling
On Wed, Feb 19, 2014 at 01:07:49PM +0200, Florin Avram wrote:
> Well, this design corrupts the working copy and I cannot use it after. I
> doubt this is what you want...
> I ran into this issue by testing how "svn revert" behaves on different
> strange situations, but, at least theoretically, anybody can run into it.

The working copy is perfectly fine.

A corrupt working copy is one that has a problem a user cannot recover
from without getting a new checkout. But in your case, removing the
obstructing file is all that needs to be done to fix the problem.

The on-disk tree is broken because you moved a file where a directory
should be *without telling svn about it*. This violates Subversion's
usage model, which is very explicit about these things. Some other
systems work differently, but with Subversion you simply have to use svn
commands for all on-disk tree modifications. If you don't, well then
you end up with problems like this.

'svn revert' tries to fix up any discrepancies between the on-disk
state and the working copy's meta data. It tries to ensure that the
on-disk state matches what the working copy meta-data indicates.
In your case it cannot do so without overwriting an unversioned file.
Subversion never deletes unversioned data voluntarily, because it
cannot know if the data is important and if a backup copy exists.
Imagine 'svn revert' overwrote an important file, and you had no backup
of it. You'd probably and rightly complain about that!
We generally don't want Subversion to delete unversioned user data.

Which is why Subversion calls this an 'obstruction'. The status output:

~ directory
! directory/file

indicates exactly what Subversion thinks of this situation.
See 'svn help status':

  '!' item is missing (removed by non-svn command) or incomplete
  '~' versioned item obstructed by some item of a different kind

So, just move your file to a safe place, and run 'svn revert' again.


Re: "svn revert" corrupts working copy

2014-02-19 Thread Philip Martin
Stefan Sperling  writes:

> So, just move your file to a safe place, and run 'svn revert' again.

In this case it's 'svn cleanup' after removing the obstruction.

I suppose Subversion could check for such obstructions before creating
the workqueue item and fail with some sort of "cannot revert" error.
Any check like that would have to happen after the outcome of earlier
reverts is known:

  svn revert wc/dir wc/dir/file

if the revert of wc/dir is successful then any obstruction of wc/dir has
been removed by the time we attempt to revert wc/dir/file.

-- 
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*


Re: "svn revert" corrupts working copy

2014-02-19 Thread Stefan Sperling
On Wed, Feb 19, 2014 at 12:17:06PM +, Philip Martin wrote:
> Stefan Sperling  writes:
> 
> > So, just move your file to a safe place, and run 'svn revert' again.
> 
> In this case it's 'svn cleanup' after removing the obstruction.

Then I misunderstood the original problem report.

Can you give me a reproduction recipe? Here's what I did:

$ ls epsilon
zeta
$ rm -rf epsilon
$ echo > epsilon
$ svn st
~   epsilon
!   epsilon/zeta
$ svn revert epsilon/zeta   
svn: E155009: Failed to run the WC DB work queue associated with 
'/tmp/svn-sandbox/trunk/epsilon/zeta', work item 5 (file-install epsilon/zeta 1 
0 1 1)
svn: E20: Can't move '/tmp/svn-sandbox/trunk/.svn/tmp/svn-6SoBLy' to 
'/tmp/svn-sandbox/trunk/epsilon/zeta': Not a directory
$ svn cleanup
svn: E155009: Failed to run the WC DB work queue associated with 
'/tmp/svn-sandbox/trunk', work item 5 (file-install epsilon/zeta 1 0 1 1)
svn: E20: Can't move '/tmp/svn-sandbox/trunk/.svn/tmp/svn-Sm3yxt' to 
'/tmp/svn-sandbox/trunk/epsilon/zeta': Not a directory
$ mv epsilon outoftheway
$ svn cleanup
$ svn st
?   outoftheway
$ svn --version
svn, version 1.8.5 (r1542147)
   compiled Jan 30 2014, 02:45:04 on x86_64-unknown-openbsd5.5


Re: "svn revert" corrupts working copy

2014-02-19 Thread Florin Avram
I think I know enough about Subversion and work with it for more than 5 
years, so I know very well what obstructed and missing means, and that 
Subversion tries to not erase non-versioned data.I know only less about 
new SVN 1.8.x behavior, which I'm testing currently, and which in some 
situations is not to explicit and it does not help a common user (even 
if comparing it with older versions, many commands became powerful, 
especially with the new SVN 1.7 architecture). Not all users work with 
an SVN client or with the command-line client directlyand/or often- 
there are infinite use-cases and tools that work over a working copy 
non-aware of SVN. The client should help, as much as possible, the users 
to fix things when they switch from their development tool to SVN.And I 
say this from my experience, after interacting with many users that 
found themselves in the situation of not knowing what to do further at a 
given point, so any information pointing to the correct cause in solving 
any issue is essential.Users are error prone and may not have enough 
experience with Subversion - help them.


I understand your point, but I have the feeling somewhere there is not a 
correct decision and things can be improved.


What I'm trying to say is that what Subversion does in this case is not 
too obvious. Think to the following situation:

- wc_root_dir
-- dir1/file.txt
-- dir2/file.txt

Now, dir2 is obstructed and a user wants to revert dir1/file.txt. By 
mistake, he can type dir2/file.txt, so:

- svn gives error and points to "svn cleanup"
- the user tries "svn cleanup" and it does not work;
- then, it tries "svn status" to see what are the states of the items, 
this way he can figure out if there is something wrong. It does not work 
also.


The errors provided by Subversion do not point to the fact that there is 
an obstructed item and the obstructing item must be removed.
What I reported previously was an information I already knew (the 
directory being obstructed), but given current situation, the user 
doesn't know that dir2 is obstructed (he wanted to work with dir1) and 
he cannot find out this using any Subversion client (since SVN givesa 
useless error).


When the client states "The parameter is incorrect", this is useless 
information for a common user (it could be some library internal 
error?!).I remember that, in SVN 1.6.x, for some similar situations, 
there were some messages like"found a file instead of a versioned 
directory" or so - this is something useful, that points to the correct 
problem. But, as long as, in the given situation, the user cannot run 
"cleanup" or "status" to know more about what is happening, the next 
idea it has is that the working copy is corrupted and it needs to check 
out again.


I agree with Philip Martin - before doing any changes to the working 
copy (and block it, from my point of view, after the example situation) 
it should do some checks and warn the user that the operation is not 
possible. This allows to work with the working copy further (runt other 
commands) and fix the issue.


Regards,
Florin


On 19.02.2014 14:36, Stefan Sperling wrote:

On Wed, Feb 19, 2014 at 12:17:06PM +, Philip Martin wrote:

Stefan Sperling  writes:


So, just move your file to a safe place, and run 'svn revert' again.

In this case it's 'svn cleanup' after removing the obstruction.

Then I misunderstood the original problem report.

Can you give me a reproduction recipe? Here's what I did:

$ ls epsilon
zeta
$ rm -rf epsilon
$ echo > epsilon
$ svn st
~   epsilon
!   epsilon/zeta
$ svn revert epsilon/zeta
svn: E155009: Failed to run the WC DB work queue associated with 
'/tmp/svn-sandbox/trunk/epsilon/zeta', work item 5 (file-install epsilon/zeta 1 
0 1 1)
svn: E20: Can't move '/tmp/svn-sandbox/trunk/.svn/tmp/svn-6SoBLy' to 
'/tmp/svn-sandbox/trunk/epsilon/zeta': Not a directory
$ svn cleanup
svn: E155009: Failed to run the WC DB work queue associated with 
'/tmp/svn-sandbox/trunk', work item 5 (file-install epsilon/zeta 1 0 1 1)
svn: E20: Can't move '/tmp/svn-sandbox/trunk/.svn/tmp/svn-Sm3yxt' to 
'/tmp/svn-sandbox/trunk/epsilon/zeta': Not a directory
$ mv epsilon outoftheway
$ svn cleanup
$ svn st
?   outoftheway
$ svn --version
svn, version 1.8.5 (r1542147)
compiled Jan 30 2014, 02:45:04 on x86_64-unknown-openbsd5.5





SVN import

2014-02-19 Thread Somashekarappa, Anup (CWM-NR)


Hi,

We are trying to build the job in jenkins where it is trying to commit to SVN 
repository.

The artifacts are getting committed successfully but we are seeing the below 
error in th output.

svncommit:
 [echo] Commiting build v031_b84_r45185_DBOS-DTR-DB to 
http://SVNURL/svndata/path/builds/v031_b84_r45185_DBOS-DTR-DB
  [svn]  started ...
  [svn] svn: database is locked
  [svn] svn: MERGE of '/svndata/path/builds': 409 Conflict 
(http://SVNURL)
  [svn]  failed !

BUILD FAILED
F:\PATH\commonDbBuild\ant_svn.xml:36: Can't import




Thanks & Regards,
Anup T S


__

This email is intended only for the use of the individual(s) to whom it is 
addressed and may be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in 
error, please advise immediately
and delete the original message. This message may have been altered without 
your or our knowledge
and the sender does not accept any liability for any errors or omissions in the 
message.

Emails are monitored by supervisory personnel in jurisdictions where monitoring 
is permitted. 
Such communications are retained and may be produced to regulatory authorities 
or others with legal rights to the information.


Re: "svn revert" corrupts working copy

2014-02-19 Thread Stefan Sperling
On Wed, Feb 19, 2014 at 02:52:48PM +0200, Florin Avram wrote:
> I understand your point, but I have the feeling somewhere there is not a
> correct decision and things can be improved.
> 
> What I'm trying to say is that what Subversion does in this case is not too
> obvious. Think to the following situation:
> - wc_root_dir
> -- dir1/file.txt
> -- dir2/file.txt

> When the client states "The parameter is incorrect", this is useless
> information for a common user (it could be some library internal error?!).

I agree here. The error messages that result from this problem are
definitely not user friendly.

This is not a new problem. Subversion has always had problems with
many of its error messages being far from understandable for users :(
I actually gave a conference talk on this very subject once, slides at
http://www.elegosoft.com/files/Downloads/Subversion_Day_2011/svn-day-berlin-2011_sperling_subversion-error-messages-demystified.pdf

In your case, "The parameter is incorrect" is most likely coming from
the Windows operating system. I'm seeing "is not a directory" instead.

> I agree with Philip Martin - before doing any changes to the working copy
> (and block it, from my point of view, after the example situation) it should
> do some checks and warn the user that the operation is not possible. This
> allows to work with the working copy further (runt other commands) and fix
> the issue.

Sometimes, it's hard to do such checks upfront, because we often
don't know what problems can happen until they actually occur.
Some operations can have way too many side effects to consider upfront,
and checking upfront could hurt performance a lot in many cases.
And if we're not very careful, such checks might actually end up breaking
legitimate use cases that were overlooked when the checks were implemented.

I think a better approach would be to try to systematically improve all
error messages raised by the work queue subsystem in the working copy
library, adding hints to the likely cause of the problem. I'd fully
support a new entry in our issue tracker for such a task. Most of these
messages are beyond what a normal user should have to deal with.


Re: "svn revert" corrupts working copy

2014-02-19 Thread Florin Avram
I'm glad we found out a common point of view. A better error message, 
that indicates the cause (at least, in the provided example, it should 
be very clear what is triggering the error), would definitely help save 
some time, since no other command works to obtain other information 
about this working copy "blockage".

Indeed, I work on Windows.

Should I create a new issue regarding the error messages for this 
situation, or someone else does this?!



On 19.02.2014 16:42, Stefan Sperling wrote:

On Wed, Feb 19, 2014 at 02:52:48PM +0200, Florin Avram wrote:

I understand your point, but I have the feeling somewhere there is not a
correct decision and things can be improved.

What I'm trying to say is that what Subversion does in this case is not too
obvious. Think to the following situation:
- wc_root_dir
-- dir1/file.txt
-- dir2/file.txt
When the client states "The parameter is incorrect", this is useless
information for a common user (it could be some library internal error?!).

I agree here. The error messages that result from this problem are
definitely not user friendly.

This is not a new problem. Subversion has always had problems with
many of its error messages being far from understandable for users :(
I actually gave a conference talk on this very subject once, slides at
http://www.elegosoft.com/files/Downloads/Subversion_Day_2011/svn-day-berlin-2011_sperling_subversion-error-messages-demystified.pdf

In your case, "The parameter is incorrect" is most likely coming from
the Windows operating system. I'm seeing "is not a directory" instead.


I agree with Philip Martin - before doing any changes to the working copy
(and block it, from my point of view, after the example situation) it should
do some checks and warn the user that the operation is not possible. This
allows to work with the working copy further (runt other commands) and fix
the issue.

Sometimes, it's hard to do such checks upfront, because we often
don't know what problems can happen until they actually occur.
Some operations can have way too many side effects to consider upfront,
and checking upfront could hurt performance a lot in many cases.
And if we're not very careful, such checks might actually end up breaking
legitimate use cases that were overlooked when the checks were implemented.

I think a better approach would be to try to systematically improve all
error messages raised by the work queue subsystem in the working copy
library, adding hints to the likely cause of the problem. I'd fully
support a new entry in our issue tracker for such a task. Most of these
messages are beyond what a normal user should have to deal with.





Re: "svn revert" corrupts working copy

2014-02-19 Thread Stefan Sperling
On Wed, Feb 19, 2014 at 05:09:11PM +0200, Florin Avram wrote:
> Should I create a new issue regarding the error messages for this situation,
> or someone else does this?!

If you have time to do so that would be great. Thanks!


Re: SVN import

2014-02-19 Thread Philip Martin
"Somashekarappa, Anup (CWM-NR)"  writes:

> We are trying to build the job in jenkins where it is trying to commit
> to SVN repository.
>
> The artifacts are getting committed successfully but we are seeing the
> below error in th output.
>
> svncommit:
>  [echo] Commiting build v031_b84_r45185_DBOS-DTR-DB to 
> http://SVNURL/svndata/path/builds/v031_b84_r45185_DBOS-DTR-DB
>   [svn]  started ...
>   [svn] svn: database is locked
>   [svn] svn: MERGE of '/svndata/path/builds': 409 Conflict 
> (http://SVNURL)
>   [svn]  failed !

The "database is locked" probably refers to a failure to access the
rep-cache.db file in the repository.  The Apache error log may contain
more information.  Which version of Subversion is used on the server?

-- 
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*


RE: SVN import

2014-02-19 Thread Somashekarappa, Anup (CWM-NR)

Hi,

I have given the full access but getting the same error. 

-rwxrwxrwx1 apache apache 2069504 Feb 16 19:24 rep-cache.db


Thanks & Regards,
Anup T S

-Original Message-
From: Philip Martin [mailto:philip.mar...@wandisco.com] 
Sent: 2014, February, 19 11:16 AM
To: Somashekarappa, Anup (CWM-NR)
Cc: users@subversion.apache.org
Subject: Re: SVN import

"Somashekarappa, Anup (CWM-NR)"  writes:

> We are trying to build the job in jenkins where it is trying to commit 
> to SVN repository.
>
> The artifacts are getting committed successfully but we are seeing the 
> below error in th output.
>
> svncommit:
>  [echo] Commiting build v031_b84_r45185_DBOS-DTR-DB to 
> http://SVNURL/svndata/path/builds/v031_b84_r45185_DBOS-DTR-DB
>   [svn]  started ...
>   [svn] svn: database is locked
>   [svn] svn: MERGE of '/svndata/path/builds': 409 Conflict 
> (http://SVNURL)
>   [svn]  failed !

The "database is locked" probably refers to a failure to access the 
rep-cache.db file in the repository.  The Apache error log may contain more 
information.  Which version of Subversion is used on the server?

--
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*
__

This email is intended only for the use of the individual(s) to whom it is 
addressed and may be privileged and confidential.
Unauthorised use or disclosure is prohibited. If you receive this e-mail in 
error, please advise immediately
and delete the original message. This message may have been altered without 
your or our knowledge
and the sender does not accept any liability for any errors or omissions in the 
message.

Emails are monitored by supervisory personnel in jurisdictions where monitoring 
is permitted. 
Such communications are retained and may be produced to regulatory authorities 
or others with legal rights to the information.



issue validation: bug, Memcached+FullText cache is totally broken in svnserve 1.8.5

2014-02-19 Thread Anatoly Orlov
Hi.

Preface:
We are trying to migrate to svn 1.8.5, with very large repository(~700G), with 
heavy load(1000+ developers, 200+ test servers) since we use svn+ssh memcached 
is reasonable cache system for us.

Problem:
Enabling memcached with full text caches in version 1.8.5. causes svnserve -t 
segfault just after first cache hit to memcached.

Reason:
memcache_get lack deserialiser for memcached full texts, and uses default one, 
which returns svn_string_t* in first parameter (thru void**) at 
cache-memcache.c:206.
This value is returned as void** thru svn_cache__get at cache.c:67 to 
read_representation at temp_serializer.c:5298
read_representation interpret this void** as svn_stringbuf_t* and puts this 
value into stream baton.
baton is used in read_handler_stringbuf at in stream.c:1397 and cause crash 
(because len and data in svn_string_t and svn_stringbuf_t are effectively 
changed places)

Traceback:
#0  0x758ea932 in memcpy () from /lib64/libc.so.6
#1  0x773427cc in read_handler_stringbuf (baton=0x81ed98, buffer=, len=0x81eb38)
at /usr/include/bits/string3.h:52
#2  0x7fffed579f6d in get_dir_contents (entries_p=0x7fffdc60, 
fs=0x6451a8, noderev=0x6877a8, pool=0x81ca98)
at subversion/libsvn_fs_fs/fs_fs.c:5524
#3  svn_fs_fs__rep_contents_dir (entries_p=0x7fffdc60, fs=0x6451a8, 
noderev=0x6877a8, pool=0x81ca98)
at subversion/libsvn_fs_fs/fs_fs.c:5679
#4  0x7fffed57a47d in svn_fs_fs__rep_contents_dir_entry 
(dirent=0x7fffdcf8, fs=0x6451a8, noderev=0x6877a8, 
name=0x687940 "trunk", result_pool=0x81ca98, scratch_pool=0x81ca98) at 
subversion/libsvn_fs_fs/fs_fs.c:5727
#5  0x7fffed5686ac in svn_fs_fs__dag_dir_entry (dirent=0x7fffdcf8, 
node=0x687720, name=0x687940 "trunk", pool=0x81ca98)
at subversion/libsvn_fs_fs/dag.c:445
#6  0x7fffed5686e9 in dir_entry_id_from_node (child_p=0x7fffddd8, 
parent=0x687720, name=, 
result_pool=0x6872b8, scratch_pool=) at 
subversion/libsvn_fs_fs/dag.c:315
#7  svn_fs_fs__dag_open (child_p=0x7fffddd8, parent=0x687720, name=, result_pool=0x6872b8, 
scratch_pool=) at subversion/libsvn_fs_fs/dag.c:1164
#8  0x7fffed583a77 in open_path (parent_path_p=0x7fffde38, 
root=0x81c0b0, path=0x6874c8 "/trunk/arcadia", flags=6, 
txn_id=0x0, pool=0x6872b8) at subversion/libsvn_fs_fs/tree.c:1006
#9  0x7fffed583ecb in get_dag (dag_node_p=0x7fffde78, root=0x81c0b0, 
path=0x6874c8 "/trunk/arcadia", needs_lock_cache=0, 
pool=0x6872b8) at subversion/libsvn_fs_fs/tree.c:1213
#10 0x7fffed583fda in svn_fs_fs__node_id (id_p=0x7fffdea8, root=, path=, 
pool=0x6872b8) at subversion/libsvn_fs_fs/tree.c:1284
#11 0x7fffed58407b in node_kind (kind_p=0x7fffdefc, root=0x81c0b0, 
path=, pool=0x6872b8)
at subversion/libsvn_fs_fs/tree.c:1333
#12 svn_fs_fs__check_path (kind_p=0x7fffdefc, root=0x81c0b0, path=, pool=0x6872b8)
at subversion/libsvn_fs_fs/tree.c:1352
#13 0x0040c933 in check_path (conn=0x633000, pool=0x6872b8, 
params=, baton=0x7fffe040)
at subversion/svnserve/serve.c:2280
#14 0x7707f063 in svn_ra_svn__handle_commands2 (conn=0x633000, 
pool=, commands=, 
baton=0x7fffe040, error_on_disconnect=0) at 
subversion/libsvn_ra_svn/marshal.c:1494
#15 0x0040bc2d in serve (conn=0x633000, params=, 
pool=0x627508) at subversion/svnserve/serve.c:3678
#16 0x0040fa5f in main (argc=, argv=) at subversion/svnserve/svnserve.c:844

Could somebody confirm the issue and/or recommend workaround?

Re: issue validation: bug, Memcached+FullText cache is totally broken in svnserve 1.8.5

2014-02-19 Thread Stefan Sperling
On Wed, Feb 19, 2014 at 10:03:47PM +0400, Anatoly Orlov wrote:
> Hi.
> 
> Preface:
> We are trying to migrate to svn 1.8.5, with very large repository(~700G), 
> with heavy load(1000+ developers, 200+ test servers) since we use svn+ssh 
> memcached is reasonable cache system for us.
> 
> Problem:
> Enabling memcached with full text caches in version 1.8.5. causes svnserve -t 
> segfault just after first cache hit to memcached.
> 
> Reason:
> memcache_get lack deserialiser for memcached full texts, and uses default 
> one, which returns svn_string_t* in first parameter (thru void**) at 
> cache-memcache.c:206.
> This value is returned as void** thru svn_cache__get at cache.c:67 to 
> read_representation at temp_serializer.c:5298
> read_representation interpret this void** as svn_stringbuf_t* and puts this 
> value into stream baton.
> baton is used in read_handler_stringbuf at in stream.c:1397 and cause crash 
> (because len and data in svn_string_t and svn_stringbuf_t are effectively 
> changed places)
> 
> Traceback:
> #0  0x758ea932 in memcpy () from /lib64/libc.so.6
> #1  0x773427cc in read_handler_stringbuf (baton=0x81ed98, 
> buffer=, len=0x81eb38)
> at /usr/include/bits/string3.h:52
> #2  0x7fffed579f6d in get_dir_contents (entries_p=0x7fffdc60, 
> fs=0x6451a8, noderev=0x6877a8, pool=0x81ca98)
> at subversion/libsvn_fs_fs/fs_fs.c:5524
> #3  svn_fs_fs__rep_contents_dir (entries_p=0x7fffdc60, fs=0x6451a8, 
> noderev=0x6877a8, pool=0x81ca98)
> at subversion/libsvn_fs_fs/fs_fs.c:5679
> #4  0x7fffed57a47d in svn_fs_fs__rep_contents_dir_entry 
> (dirent=0x7fffdcf8, fs=0x6451a8, noderev=0x6877a8, 
> name=0x687940 "trunk", result_pool=0x81ca98, scratch_pool=0x81ca98) at 
> subversion/libsvn_fs_fs/fs_fs.c:5727
> #5  0x7fffed5686ac in svn_fs_fs__dag_dir_entry (dirent=0x7fffdcf8, 
> node=0x687720, name=0x687940 "trunk", pool=0x81ca98)
> at subversion/libsvn_fs_fs/dag.c:445
> #6  0x7fffed5686e9 in dir_entry_id_from_node (child_p=0x7fffddd8, 
> parent=0x687720, name=, 
> result_pool=0x6872b8, scratch_pool=) at 
> subversion/libsvn_fs_fs/dag.c:315
> #7  svn_fs_fs__dag_open (child_p=0x7fffddd8, parent=0x687720, name= optimized out>, result_pool=0x6872b8, 
> scratch_pool=) at subversion/libsvn_fs_fs/dag.c:1164
> #8  0x7fffed583a77 in open_path (parent_path_p=0x7fffde38, 
> root=0x81c0b0, path=0x6874c8 "/trunk/arcadia", flags=6, 
> txn_id=0x0, pool=0x6872b8) at subversion/libsvn_fs_fs/tree.c:1006
> #9  0x7fffed583ecb in get_dag (dag_node_p=0x7fffde78, root=0x81c0b0, 
> path=0x6874c8 "/trunk/arcadia", needs_lock_cache=0, 
> pool=0x6872b8) at subversion/libsvn_fs_fs/tree.c:1213
> #10 0x7fffed583fda in svn_fs_fs__node_id (id_p=0x7fffdea8, 
> root=, path=, 
> pool=0x6872b8) at subversion/libsvn_fs_fs/tree.c:1284
> #11 0x7fffed58407b in node_kind (kind_p=0x7fffdefc, root=0x81c0b0, 
> path=, pool=0x6872b8)
> at subversion/libsvn_fs_fs/tree.c:1333
> #12 svn_fs_fs__check_path (kind_p=0x7fffdefc, root=0x81c0b0, path= optimized out>, pool=0x6872b8)
> at subversion/libsvn_fs_fs/tree.c:1352
> #13 0x0040c933 in check_path (conn=0x633000, pool=0x6872b8, 
> params=, baton=0x7fffe040)
> at subversion/svnserve/serve.c:2280
> #14 0x7707f063 in svn_ra_svn__handle_commands2 (conn=0x633000, 
> pool=, commands=, 
> baton=0x7fffe040, error_on_disconnect=0) at 
> subversion/libsvn_ra_svn/marshal.c:1494
> #15 0x0040bc2d in serve (conn=0x633000, params=, 
> pool=0x627508) at subversion/svnserve/serve.c:3678
> #16 0x0040fa5f in main (argc=, argv= optimized out>) at subversion/svnserve/svnserve.c:844
> 
> Could somebody confirm the issue and/or recommend workaround?

Yes, please file an issue. This has clearly not been tested before release :(

As a workaround you will need to disable caching, as the in-process cache of
svnserve won't really help you with svn+ssh unfortunately.

This shouldn't be very hard to fix. But the next 1.8 release has just
been prepared so you probably won't see a patched release for a few weeks.