Re: Performance issue with svn export [svn 1.8.11]

2015-05-31 Thread Stefan Fuhrmann
From: Nouha Terzi 
Date: Tue, 26 May 2015 17:05:41 +0100

Hello all,
>
> we are deploying svn 1.8.11 on our server Rhel 6.6.
> We are facing some issue within svn export.
> it seems that it has some perf limitation when dealing with a huge number
> of files.
>
> we have this delay when doing an svn export url and the target has:
>
> less than 20k files export = ~ 1 minute
>
> more than 30k files export > 1 hour
>
> Is it a known issue? Has someone encouter thsi behavior?
>

There is a known issues that might apply: The number of
entries (immediate children) in a directory should not exceed
a few 1000, see the warning box in 1.7 release notes:

http://subversion.apache.org/docs/release-notes/1.7.html#server-performance-tuning

If all 30k files are from a single "flat" directory or is there one
large directory somewhere in this tree, you'll hit that issue.

Having directories with many 1000 entries is all kinds of bad.
Don't do it. If you really have to do it, increase the SVN server
cache size. Note that you have to quadruple the cache size
to double the maximum cache-able object size.

-- Stefan^2.


RE: svn 1.8.13 test failures: wc-queries-test.exe and move_tests.py

2015-05-31 Thread Cooke, Mark
> > -Original Message-
> > From: Cooke, Mark [mailto:mark.co...@siemens.com]
> > Sent: vrijdag 29 mei 2015 15:21
> >
> > Hello,
> >
> > I am trying to compile svn (and httpd) with VS2008 (for python 2.7
> > compatibility) on Win 7 Enterprise and am getting some test failures:
> > {{{
> >win-tests.py --release --cleanup --parallel
> >win-tests.py -r -c --httpd-dir=d:\Apache24 --httpd-daemon
> > }}}
> >
> > [40/96] wc-queries-test.exe ... FAILED
> > [72/96] move_tests.py . success
> >
> > For #40 I noticed that Andreas confirmed [1] that r1672295 fixes a
> >  (similar?) issue (which is more recent than the 1.8.13 tag), so can
> > I safely ignore this?
> 
> -Original Message-
> From: Bert Huijben [mailto:b...@qqmail.nl]
> Sent: 29 May 2015 16:15
> 
> What version of sqlite do you use,

I made a mistake and took a nightly snapshot:
- sqlite-amalgamation-201505290135.zip

> and what is the error that you see?

Apologies for not being more precise earlier, I am still finding my way around 
the code trees.  This is the summary:-

At least one test FAILED, checking D:\svn\src\Release\dav-tests.log
FAIL:  wc-queries-test.exe 3: test query expectations
Summary of test results:
  1950 tests PASSED
  55 tests SKIPPED
  32 tests XFAILED (1 WORK-IN-PROGRESS)
  1 test FAILED
SUMMARY: Some tests failed.

...and from the test log:-

START: wc-queries-test.exe
DBG: Using Sqlite 3.8.11
PASS:  wc-queries-test.exe 1: sqlite up-to-date
PASS:  wc-queries-test.exe 2: queries are parsable
svn_tests: E26: STMT_SELECT_EXTERNALS_DEFINED: Uses externals with only 1 
index component: (wc_id=? )
SELECT local_relpath, def_local_relpath FROM externals WHERE (wc_id = ?1 AND 
def_local_relpath = ?2)OR (wc_id = ?1 AND (((def_local_relpath) > (CASE 
(?2) WHEN '' THEN '' ELSE (?2) || '/' END)) AND ((def_local_relpath) < CASE 
(?2) WHEN '' THEN X'' ELSE (?2) || '0' END))) 
svn_tests: E200035: Additional errors:
svn_tests: E200035: |SEARCH TABLE externals USING COVERING INDEX 
I_EXTERNALS_DEFINED (wc_id=?)
FAIL:  wc-queries-test.exe 3: test query expectations
PASS:  wc-queries-test.exe 4: test schema statistics
END: wc-queries-test.exe
ELAPSED: wc-queries-test.exe 0:00:00.125000

For the `move_tests.py` crash "success" the log states:

START: move_tests.py
PASS:  move_tests.py 1: lateral (rename) move of a file test
PASS:  move_tests.py 2: sibling move of a file test
PASS:  move_tests.py 3: shallower move of a file test
PASS:  move_tests.py 4: deeper move of a file test
PASS:  move_tests.py 5: test property merging on move-update
PASS:  move_tests.py 6: move a missing directory
PASS:  move_tests.py 7: move many and delete-on-update
W: svn: E235000: In file '..\..\..\subversion\libsvn_wc\wc_db_update_move.c' 
line 809: assertion failed (move_dst_revision == expected_move_dst_revision || 
status == svn_wc__db_status_not_present)
W: 
W: This application has requested the Runtime to terminate it in an unusual way.
W: Please contact the application's support team for more information.
W: CWD: D:\svn\src\Release\subversion\tests\cmdline
W: EXCEPTION: Failure: Command failed: 
"D:\svn\src\Release\subversion\svn\svn.exe up 
svn-test-work\working_copies\move_tests-8 --accept ..."; exit code 3
Traceback (most recent call last):
  File "d:\svn\src\subversion\tests\cmdline\svntest\main.py", line 1561, in run
rc = self.pred.run(sandbox)
  File "d:\svn\src\subversion\tests\cmdline\svntest\testcase.py", line 114, in 
run
return self._delegate.run(sandbox)
  File "d:\svn\src\subversion\tests\cmdline\svntest\testcase.py", line 176, in 
run
return self.func(sandbox)
  File "d:\svn\src\subversion\tests\cmdline\move_tests.py", line 1397, in 
move_many_update_add
wc_dir, '--accept', 'mine-conflict')
  File "d:\svn\src\subversion\tests\cmdline\svntest\actions.py", line 865, in 
run_and_verify_update
exit_code, output, errput = main.run_svn(error_re_string, 'up', *args)
  File "d:\svn\src\subversion\tests\cmdline\svntest\main.py", line 682, in 
run_svn
*(_with_auth(_with_config_dir(varargs
  File "d:\svn\src\subversion\tests\cmdline\svntest\main.py", line 365, in 
run_command
None, *varargs)
  File "d:\svn\src\subversion\tests\cmdline\svntest\main.py", line 557, in 
run_command_stdin
'"; exit code ' + str(exit_code))
Failure: Command failed: "D:\svn\src\Release\subversion\svn\svn.exe up 
svn-test-work\working_copies\move_tests-8 --accept ..."; exit code 3
XFAIL: move_tests.py 8: move many and add-on-update
PASS:  move_tests.py 9: delete moved node, still a move
PASS:  move_tests.py 10: copy, move and commit
PASS:  move_tests.py 11: move to and from an external
PASS:  move_tests.py 12: move conflict markers
END: move_tests.py
ELAPSED: move_tests.py 0:01:02.563000

...which I now see is an XFAIL ~ but is the test still valid with the crash?

> In most cases you can ignore the wc-query-expectations error failure, which
> only signals some performance loss. But it is hard to tell what failure