from:"pierre.viret"

Segmentation Fault with SVN Client related to serf

2015-01-05 Thread pierre.viret

Hello

We have following problem with the SVN native clients in our environment: the 
client crashes with a memory access violation error while executing an "ls" 
command. We have reproduced the error on debian with SVN 1.8.10 and on Windows 
with the latest SVN 1.8.11. With older versions of SVN client (SlikSvn 1.6.x / 
1.7.x or TurtoiseSVN 1.7.x on windows) we never get this error if we use the 
neon module but we get it with the serf module. Note that with SVNKit we have 
no problem. So it seems that the error is related to serf.

The problem must be related to some timing issue: if we have debugging turned 
on in our Proxy the answers of the servers are slower and we get the error more 
often. Actually we get the error maybe 30% of the times we try to execute the 
ls command. 

Has anyone else experienced the same problem or some ideas how we could solve 
it? Should we fill a bug report, as a segmentation fault is surely not to be 
expected ?

Regards,
Pierre


Here the output on windows:

C:\seu\SlikSvn\bin>svn --version
svn, Version 1.8.11-SlikSvn-1.8.11-X64 (SlikSvn/1.8.11) X64
   übersetzt am Dec  9 2014, um 13:44:31 auf x86_64-microsoft-windows6.2.9200
 
Copyright (C) 2014 The Apache Software Foundation.
Diese Software besteht aus Beiträgen vieler Personen;
siehe Datei NOTICE für weitere Informationen.
Subversion ist Open Source Software, siehe http://subversion.apache.org/
 
Die folgenden ZugriffsModule (ZM) für Projektarchive stehen zur Verfügung:
 
* ra_svn : Modul zum Zugriff auf ein Projektarchiv über das 
svn-Netzwerkprotokoll.
  - mit Cyrus-SASL-Authentifizierung
  - behandelt Schema »svn«
* ra_local : Modul zum Zugriff auf ein Projektarchiv auf der lokalen Festplatte
  - behandelt Schema »file«
* ra_serf : Modul zum Zugriff auf ein Projektarchiv über das Protokoll WebDAV 
mittels serf.
  - verwendet serf 1.3.8
  - behandelt Schema »http«
  - behandelt Schema »https«
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
(crashed)
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
This application has halted due to an unexpected error.
A crash report and minidump file were saved to disk, you can find them here:
C:\Users\viretp\AppData\Local\Temp\svn-crash-log20150106074548.log
C:\Users\viretp\AppData\Local\Temp\svn-crash-log20150106074548.dmp
Please send the log file to users@subversion.apache.org to help us analyze
and solve this problem.
 
NOTE: The crash report and minidump files can contain some sensitive information
(filenames, partial file content, usernames and passwords etc.)
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
(crashed)
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
branches/
tags/
trunk/
 
C:\seu\SlikSvn\bin>svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
(crashed)
 
Here the content of the error log file:
Process info:
Cmd line: svn  ls https://127.0.0.1:7771/svn/t_sponis_testrepo
Working Dir: C:\seu\SlikSvn\bin
Version:  1.8.11-SlikSvn-1.8.11-X64 (SlikSvn/1.8.11) X64, compiled Dec  9 2014, 
13:44:31
Platform: Windows OS version 6.1 build 7601 Service Pack 1

Exception: ACCESS_VIOLATION

Registers:
Rax=0039 Rcx=022af660 Rdx=09cd 
Rbx=022c9828
Rsp=0014f0f8 Rbp=022b0768 Rsi= 
Rdi=022ba1f5
R8= fffe5ff8 R9= 31045bd5b000 R10= 00ffeeffee01017e 
R11=022c9828
R12= R13=0065c9c4 R14=0014f2d0 
R15=
cs=0033  ss=002b  ds=002b  es=0053  fs=002b  gs=002b  ss=0094

Stacktrace:
#1  0x74bcc24d in memmove()
#2  0x7fef4d2708f in (unknown function)
#3  0x7fef4d21abe in (unknown function)
#4  0x7fef4d21f1a in (unknown function)
#5  0x7fef4d22341 in (unknown function)
#6  0x7fef4d289d8 in (unknown function)
#7  0x7fef4d208b7 in (unknown function)
#8  0x7fef4d20ada in (unknown function)
#9  0x7fef4d1ee8a in (unknown function)
#10  0x7fef4d1efe5 in (unknown function)
#11  0x7fef4d54d30 in svn_ra_svn_init()
#12  0

AW: Segmentation Fault with SVN Client related to serf

2015-01-06 Thread pierre.viret

> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Dienstag, 6. Januar 2015 12:49
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: Segmentation Fault with SVN Client related to serf
> 
>  writes:
> 
> > viretp@mwbox:~$ svn ls https://127.0.0.1:7771/svn/t_sponis_testrepo
> > Segmentation fault
> >
> > In /var/log/syslog:
> > Jan  5 14:31:23 mwbox kernel: [14750.072747] svn[21822]: segfault at
> > 7f3733f39000 ip 7f37328ae3cf sp 7fff9e3df468 error 7 in
> > libc-2.13.so[7f373278a000+182000]
> >
> >
> > Here the stacktrace with gdb:
> >
> > (gdb) bt
> > #0  __memcpy_ssse3 () at
> > ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:2833
> > #1  0x735f6fa3 in serf_bstrmemdup () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #2  0x735faecb in ?? () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #3  0x735fb2d4 in serf_bucket_response_status () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #4  0x735fe219 in serf__handle_auth_response () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #5  0x735f4f29 in serf__process_connection () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #6  0x735f38ae in serf_event_trigger () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #7  0x735f39d9 in serf_context_run () from
> > /usr/lib/x86_64-linux-gnu/libserf-1.so.1
> > #8  0x7573b522 in svn_ra_serf__context_run_wait () from
> > /usr/lib/x86_64-linux-gnu/libsvn_ra_serf-1.so.1
> 
> What sort of authentication are you using on your proxy and webserver?
> Is there authentication for the proxy?  Is the client handling proxy or apache
> authentication when it crashed?  Are you using one of the password stores?  It
> might be helpful to look at the network traffic although this can be tricky 
> with https.
> One way that may work is to use a socat proxy:
> 
>socat -v TCP-LISTEN:9630,reuseaddr,fork OPENSSL:127.0.0.1:7771,verify=0
> 
> and then use http://localhost:9630/ instead of https://localhost:7771/ in your
> Subversion command. You may need to use TCP6-LISTEN.
> 
> --
> Philip Martin | Subversion Committer
> WANdisco // *Non-Stop Data*

Here a description of the architecture we are using:
- a http-proxy is running on 127.0.0.1:7771 which builds a secure https tunnel 
to a special reverse proxy, using a certificate from a smart-card
- the reverse proxy then connects to the apache server used as subversion server
- the apache server uses a proprietary module for authentication using a 
security token placed in the http header.
- there is no authentication between the svn client and the local http-proxy
- the svn client is not aware that it connects through a proxy

I have tried using http instead of https for the connection between the svn 
client and the local proxy but in this case we get the segmentation fault by 
each call and not only 30% of the times...
The stacktrace looks very similar (I have pasted it at the end of the mail).

About network traffic: I have analysed the traffic between the local http-proxy 
and the svn client because I had supposed that some packet length was wrongly 
set, but could not find any clue about the reason for the segmentation fault. I 
have compared the traffic (with a svn client 1.6) between using neon and using 
serf but could not find anything explaining the segmentation fault in the case 
we use serf.

viretp@mwbox:~/work/APLAT-105/tests/debian$ gdb svn
GNU gdb (GDB) 7.4.1-debian
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/bin/svn...(no debugging symbols found)...done.
(gdb) run ls http://127.0.0.1:7771/svn/t_sponis_testrepo
Starting program: /usr/bin/svn ls http://127.0.0.1:7771/svn/t_sponis_testrepo
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
__memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:2833
2833../sysdeps/x86_64/multiarch/memcpy-ssse3.S: No such file or directory.
(gdb) bt
#0  __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:2833
#1  0x735f6fa3 in serf_bstrmemdup () from 
/usr/lib/x86_64-linux-gnu/libserf-1.so.1
#2  0x735faecb in ?? () from /usr/lib/x86_64-linux-gnu/libserf-1.so.1
#3  0x735fb2d4 in serf_bucket_response_status () from 
/usr/lib/x86_64-linux-gnu/libserf-1.so.1
#4  0x735fe219 in serf__handle_auth_response () from 
/usr/lib/x86_64-linux-gnu/libserf-1.so.1
#5  0x735f4f29 in se

AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-06 Thread pierre.viret



> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Dienstag, 6. Januar 2015 15:02
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: Segmentation Fault with SVN Client related to serf
> 
[...]
> 
> It possible that something in the final request is causing the problem but 
> it's also
> possible that something in one of the earlier requests is not handled 
> correctly and
> that is the cause of the problem.  So we really need to know the whole 
> sequence
> of requests that get sent.  I don't know how much debugging you can do with 
> the
> Debian binary, it might be easier if you install the libserf1-dbg package.  
> This is the
> code that appears to be crashing:
> 
[...]

You asked for the whole sequence of request, here is the whole tracing of our 
proxy: I hope this helps. But as already said: the svn client does not crash 
always but 30% of the times we execute exactly the same ls command.

2015-01-06 15:43:24,111 DEBUG ProxyHttpHandler - >OPTIONS 
/svn/t_sponis_testrepo HTTP/1.1
2015-01-06 15:43:24,111 DEBUG ProxyHttpHandler - Opening connection to URL: 
https://tpfesa101.pnet.ch:443/svn/t_sponis_testrepo
2015-01-06 15:43:24,112 TRACE DAVHttpsURLConnection - Creating for 
https://tpfesa101.pnet.ch:443/svn/t_sponis_testrepo
2015-01-06 15:43:24,112 DEBUG DAVHttpsURLConnection - setRequestMethod(OPTIONS)
2015-01-06 15:43:24,112 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning GET
2015-01-06 15:43:24,112 DEBUG DAVHttpsURLConnection - replace method value 
'GET' in internal delegate with new value 'OPTIONS'
2015-01-06 15:43:24,114 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:43:24,114 DEBUG ProxyHttpHandler - Size of request headers: 7
2015-01-06 15:43:24,114 DEBUG ProxyHttpHandler - >Content-type: text/xml
2015-01-06 15:43:24,114 DEBUG ProxyHttpHandler - >Host: 127.0.0.1:7771
2015-01-06 15:43:24,114 DEBUG ProxyHttpHandler - >Accept-encoding: gzip
2015-01-06 15:43:24,114 DEBUG ProxyHttpHandler - >Content-length: 131
2015-01-06 15:43:24,114 DEBUG ProxyHttpHandler - >Dav: 
http://subversion.tigris.org/xmlns/dav/svn/depth,http://subversion.tigris.org/xmlns/dav/svn/mergeinfo,http://subversion.tigris.org/xmlns/dav/svn/log-revprops
2015-01-06 15:43:24,115 DEBUG ProxyHttpHandler - >Connection: keep-alive
2015-01-06 15:43:24,115 DEBUG ProxyHttpHandler - >User-agent: SVN/1.8.10 
(x86_64-pc-linux-gnu) serf/1.3.7
2015-01-06 15:43:24,115 DEBUG ProxyHttpHandler - Connecting to server...
2015-01-06 15:43:24,115 TRACE DAVHttpsURLConnection - connect()
2015-01-06 15:43:24,115 DEBUG SslSocketFactory - Creating an unconnected 
SSLSocket()
2015-01-06 15:43:24,130 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:43:24,131 DEBUG DAVHttpsURLConnection - getOutputStream() with 
method == OPTIONS
2015-01-06 15:43:24,131 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:43:24,131 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:43:24,131 DEBUG DAVHttpsURLConnection - replace method value 
'OPTIONS' in internal delegate with new value 'TRACE'
2015-01-06 15:43:24,132 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning TRACE
2015-01-06 15:43:24,132 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning TRACE
2015-01-06 15:43:24,132 DEBUG DAVHttpsURLConnection - replace method value 
'TRACE' in internal delegate with new value 'OPTIONS'
2015-01-06 15:43:24,132 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:43:24,132 TRACE ProxyHttpHandler - Read 131 bytes from client
2015-01-06 15:43:24,133 TRACE ProxyHttpHandler - >
2015-01-06 15:43:24,133 DEBUG ProxyHttpHandler - Total number of bytes read 
from client: 131
2015-01-06 15:43:24,133 DEBUG AplatCookieHandler - Retrieve all cookies for 
URI: https://tpfesa101.pnet.ch:443/svn/t_sponis_testrepo
2015-01-06 15:43:24,133 DEBUG AplatCookieHandler - Append cookie: 
Navajo=gIb1uQ8SXfzepKGZcXFbI+zNrwxwqAIM+CUI4LudfRTv09laT7Q6NBLoYO/gwjS9TUtIvk8ia94-
2015-01-06 15:43:24,259 DEBUG AplatCookieHandler - save cookies from URI: 
https://tpfesa101.pnet.ch:443/svn/t_sponis_testrepo
2015-01-06 15:43:24,259 DEBUG ProxyHttpHandler - <200 OK
2015-01-06 15:43:24,260 DEBUG ProxyHttpHandler - Size of response headers: 23
2015-01-06 15:43:24,260 DEBUG ProxyHttpHandler -

AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-07 Thread pierre.viret

> -Ursprüngliche Nachricht-
> Von: Stefan Sperling [mailto:s...@elego.de]
> Gesendet: Dienstag, 6. Januar 2015 15:59
> An: Philip Martin
> Cc: Viret Pierre, PF54; users@subversion.apache.org
> Betreff: Re: AW: Segmentation Fault with SVN Client related to serf
> 
> On Tue, Jan 06, 2015 at 02:01:52PM +, Philip Martin wrote:
> > Typically the client will first send an OPTIONS request and get back a
> > 401 or a 200.  The client will then send further OPTIONS and PROPFIND
> > requests.  Can you identify which request and response is causing the
> > crash?  Also which, if any, requests are processed without problem
> > before the one that causes the crash?
> 
> I would be interested in knowing which end-of-line terminators each line in 
> the auth
> request header is using.
> Are some lines terminated with LF, and some with CRLF?

I'm not sure what you mean about auth request because in our case the svn 
client does not use authentication. How can I find the header you mean? What is 
it's name?

Remarque concernant la sécurité:
Ce courriel provenant de PostFinance est signé. Vous trouverez d'autres 
informations à ce sujet sous: 
https://www.postfinance.ch/e-signature.
Ne divulguez jamais vos éléments de sécurité à des tiers.

smime.p7s
Description: S/MIME Cryptographic Signature

AW: AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-07 Thread pierre.viret



> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Mittwoch, 7. Januar 2015 14:51
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: AW: Segmentation Fault with SVN Client related to serf
> 
[...]
> This is a bit confusing:
> 
>   2015-01-06 15:43:24,260 DEBUG ProxyHttpHandler -    2015-01-06 15:43:24,261 DEBUG ProxyHttpHandler - Do not send null key back
> to client null: HTTP/1.1 200 OK
> 
> That's the HTTP status line so why the message about not sending it back to 
> the
> client?  I'd expect everything to break if the proxy doesn't send the status 
> line
> back.  Is the proxy really blocking the status line?
> It's hard to believe so I think the message must mean something else.
> If you were to use a socat proxy between the client and your main proxy we 
> could
> see exactly what the client is receiving.
> 
> I see the client sending 2 OPTIONS requests which is typical for a modern 
> client
> negotiating Subversion's v2 protocol.  The first server response contains 
> SVN-Me-
> Resource indicating a modern server that supports v2, but the second response
> does not include it.  Why did the server stop supporting v2?  Is there some 
> sort of
> load balancer sending requests to multiple servers, or some sort of HTTP 
> caching?
> Almost all the headers are missing from the second OPTIONS repsonse.
> 
> The client goes on to send v1 PROPFIND requests to !svn/vcc/default and
> !svn/bln, so the second OPTIONS response appears to have prevented the
> v2 protocol being used.
> 
> It's still not clear to me why the client fails but it may be hard to 
> reproduce outside
> of your environment given the strange behaviour of the server.
> 
> 
The status line is not blocked. The debugging message means that we have got an 
header with a null key and with the status line as value: in this case the 
header is not added to the request sent back to the client. This seems to be 
common that some HTTP Implementations return such a null key header in the list 
of headers of a request.

I don't think that we would get more informations using socat because our http 
proxy should debug everything it gets, but I could give it a try.

We are using subversion server version 1.8.10: maybe is the behavior there 
other than with older versions?

I send you here the debugging output in case the "svn ls" command runs without 
crash: this could help to understand the problem.

2015-01-06 15:36:28,358 TRACE DAVHttpsURLConnection - Creating for 
https://tpfesa101.pnet.ch:443/svn/t_sponis_testrepo
2015-01-06 15:36:28,358 DEBUG DAVHttpsURLConnection - setRequestMethod(OPTIONS)
2015-01-06 15:36:28,358 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning GET
2015-01-06 15:36:28,359 DEBUG DAVHttpsURLConnection - replace method value 
'GET' in internal delegate with new value 'OPTIONS'
2015-01-06 15:36:28,359 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:36:28,359 DEBUG ProxyHttpHandler - Size of request headers: 7
2015-01-06 15:36:28,359 DEBUG ProxyHttpHandler - >Content-type: text/xml
2015-01-06 15:36:28,359 DEBUG ProxyHttpHandler - >Host: 127.0.0.1:7771
2015-01-06 15:36:28,359 DEBUG ProxyHttpHandler - >Accept-encoding: gzip
2015-01-06 15:36:28,359 DEBUG ProxyHttpHandler - >Content-length: 131
2015-01-06 15:36:28,360 DEBUG ProxyHttpHandler - >Dav: 
http://subversion.tigris.org/xmlns/dav/svn/depth,http://subversion.tigris.org/xmlns/dav/svn/mergeinfo,http://subversion.tigris.org/xmlns/dav/svn/log-revprops
2015-01-06 15:36:28,360 DEBUG ProxyHttpHandler - >Connection: keep-alive
2015-01-06 15:36:28,360 DEBUG ProxyHttpHandler - >User-agent: SVN/1.8.10 
(x86_64-pc-linux-gnu) serf/1.3.7
2015-01-06 15:36:28,360 DEBUG ProxyHttpHandler - Connecting to server...
2015-01-06 15:36:28,360 TRACE DAVHttpsURLConnection - connect()
2015-01-06 15:36:28,360 DEBUG SslSocketFactory - Creating an unconnected 
SSLSocket()
2015-01-06 15:36:28,373 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:36:28,373 DEBUG DAVHttpsURLConnection - getOutputStream() with 
method == OPTIONS
2015-01-06 15:36:28,373 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:36:28,373 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:36:28,374 DEBUG DAVHttpsURLConnection - replace method value 
'OPTIONS' in internal delegate with new value 'TRACE'
2015-01-06 15:36:28,374 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning TRACE
2015-01-06 15:36:28,374 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning TRACE
2015-01-06 15:36:28,374 DEBUG DAVHttpsURLConnection - replace method value 
'TRACE' in internal delegate with new value 'OPTIONS'
2015-01-06 15:36:28,374 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-06 15:36:28,374 TRACE ProxyHttpHandler - Read 131 bytes from c

AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-08 Thread pierre.viret



> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Donnerstag, 8. Januar 2015 14:34
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: AW: AW: Segmentation Fault with SVN Client related to serf
> 
[...]
> 
> I've never seen Apache send a second status line. Perhaps the message about 
> null
> key applies to this mysterious line in the trace:
> 

I suppose that this is some specifical java implementation. I simply get a 
header with key null and value = status line when I get the list of headers 
from the HttpsURLConnection from java, and I do not include it in the headers 
sent back to the client.
See ...

> ch.nevis.session.secroles: 
> auth.strong,idma_Benutzer,infplat_user,esclienttest_r
> oleBC,esclienttest_roleA,esclienttest_roleB,depo_Admin,itam_Leser,itam_Administ
> r
> ator,itsms_asset_viewer,itsms_svc_req_usr,testsaml_admin,global.allow,acc.idma,
> a
> cc.infplat,acc.esclienttest,acc.depo,acc.itam,acc.itsms,acc.testsaml
> 

This header is added by the EntryServer and contains the set of roles of the 
user, but our HttpProxy does not use it.

> I see that every response also has:
> 
>   Connection: close
> 
> This looks as if you do not have KeepAlive enabled on the server, or perhaps 
> the
> proxy does not support pipelining.  That is going to result in poor 
> performance for
> clients that use serf.
> 

I'm not sure what you mean with "pipelining". I would expect that we have 
KeepAlive enabled but I should check this.

> Using a standard 1.8 client against this dummy server gives:
> 
> $ svn ls http://127.0.0.1:/svn/t_sponis_testrepo
> svn: E175009: XML Parsing failed: Unexpected root element 'multistatus'
> 
[...]
> Your client will not have the debug code to do that.  How is your client 
> switching to
> the v1 protocol?  Either your proxy is not forwarding the SVN-Me-Resource 
> header
> or your client is patched to ignore v2, which ever it is your environment is 
> non-
> standard.  How does your client respond when run against this dummy server?
> (You must restart the dummy server after a client fail so that responses 
> match the
> requests.)

I got exactly the same result using your dummy server with my client.

I have captured the http traffic, you will find it in the attachment 
crash_http_capture.pcap. The debugging output of the HttpProxy is in the 
attachment crash_http.txt, it differs from the output when using https.
I had to switch from https to http for this. Please note that the crash occurs 
not exactly at the same time with http, and that we get the segmentation fault 
each time we run "ls" and not 30% of the time like with https: this could be an 
interesting point for the analyze.

The subversion server version we use is 1.8.10.

Regards,
Pierre

Remarque concernant la sécurité:
Ce courriel provenant de PostFinance est signé. Vous trouverez d'autres 
informations à ce sujet sous: 
https://www.postfinance.ch/e-signature.
Ne divulguez jamais vos éléments de sécurité à des tiers.2015-01-09 07:18:59,922 DEBUG HttpProxy - Listening on http://127.0.0.1:7771
2015-01-09 07:19:12,155 DEBUG ProxyHttpHandler - >OPTIONS 
/svn/t_sponis_testrepo HTTP/1.1
2015-01-09 07:19:12,155 DEBUG ProxyHttpHandler - Opening connection to URL: 
https://tpfesa101.pnet.ch:443/svn/t_sponis_t
estrepo
2015-01-09 07:19:12,157 TRACE DAVHttpsURLConnection - Creating for 
https://tpfesa101.pnet.ch:443/svn/t_sponis_testrepo
2015-01-09 07:19:12,157 DEBUG DAVHttpsURLConnection - setRequestMethod(OPTIONS)
2015-01-09 07:19:12,160 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning GET
2015-01-09 07:19:12,160 DEBUG DAVHttpsURLConnection - replace method value 
'GET' in internal delegate with new value 'OP
TIONS'
2015-01-09 07:19:12,160 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-09 07:19:12,162 DEBUG ProxyHttpHandler - Size of request headers: 7
2015-01-09 07:19:12,162 DEBUG ProxyHttpHandler - >Content-type: text/xml
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - >Host: 127.0.0.1:7771
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - >Accept-encoding: gzip
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - >Content-length: 131
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - >Dav: 
http://subversion.tigris.org/xmlns/dav/svn/depth,http://subve
rsion.tigris.org/xmlns/dav/svn/mergeinfo,http://subversion.tigris.org/xmlns/dav/svn/log-revprops
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - >Connection: keep-alive
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - >User-agent: 
SVN/1.8.11-SlikSvn-1.8.11-X64 (x64-microsoft-windows)
serf/1.3.8
2015-01-09 07:19:12,165 DEBUG ProxyHttpHandler - Connecting to server...
2015-01-09 07:19:12,165 TRACE DAVHttpsURLConnection - connect()
2015-01-09 07:19:12,165 DEBUG SslSocketFactory - Creating an unconnected 
SSLSocket()
2015-01-09 07:19:12,225 TRACE DAVHttpsURLConnection - getRequestMethod() 
returning OPTIONS
2015-01-09 07:19

AW: AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-09 Thread pierre.viret



> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Mittwoch, 7. Januar 2015 14:51
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: AW: Segmentation Fault with SVN Client related to serf
> 
>  writes:
> 
> > You asked for the whole sequence of request, here is the whole tracing
> > of our proxy: I hope this helps. But as already said: the svn client
> > does not crash always but 30% of the times we execute exactly the same
> > ls command.
> 
> This is a bit confusing:
> 
>   2015-01-06 15:43:24,260 DEBUG ProxyHttpHandler -    2015-01-06 15:43:24,261 DEBUG ProxyHttpHandler - Do not send null key back
> to client null: HTTP/1.1 200 OK
> 
> That's the HTTP status line so why the message about not sending it back to 
> the
> client?  I'd expect everything to break if the proxy doesn't send the status 
> line
> back.  Is the proxy really blocking the status line?
> It's hard to believe so I think the message must mean something else.
> If you were to use a socat proxy between the client and your main proxy we 
> could
> see exactly what the client is receiving.
> 
> I see the client sending 2 OPTIONS requests which is typical for a modern 
> client
> negotiating Subversion's v2 protocol.  The first server response contains 
> SVN-Me-
> Resource indicating a modern server that supports v2, but the second response
> does not include it.  Why did the server stop supporting v2?  Is there some 
> sort of
> load balancer sending requests to multiple servers, or some sort of HTTP 
> caching?
> Almost all the headers are missing from the second OPTIONS repsonse.
> 
> The client goes on to send v1 PROPFIND requests to !svn/vcc/default and
> !svn/bln, so the second OPTIONS response appears to have prevented the
> v2 protocol being used.
> 
> It's still not clear to me why the client fails but it may be hard to 
> reproduce outside
> of your environment given the strange behaviour of the server.
> 
> 

I have found following in our apache config for the subversion server:


ServerName @@httpd.ip.name1@@

SSLEngine ON

BrowserMatch ".*MSIE [2-5]\..*" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0

###replace:expand_vhhttps###


This could explain the nokeepalive and the downgrade to 1.0? I'm not sure why 
this is here, I will check it with our Apache expert.

Remarque concernant la sécurité:
Ce courriel provenant de PostFinance est signé. Vous trouverez d'autres 
informations à ce sujet sous: 
https://www.postfinance.ch/e-signature.
Ne divulguez jamais vos éléments de sécurité à des tiers.

smime.p7s
Description: S/MIME Cryptographic Signature

AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-09 Thread pierre.viret



> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Freitag, 9. Januar 2015 10:48
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: AW: AW: AW: Segmentation Fault with SVN Client related to 
> serf
> 
[...]
> 
> I see why the client switches to the v1 protocol.  Your proxy is changing the 
> case
> of the HTTP headers and "SVN-Me-Resource" becomes "Svn-me-resource".  That
> has uncovered a bug in the client: it does a case-sensitive check for "SVN" 
> before
> doing a case-insensitive check for "svn-me-resource".
> 

Yes you are right, I have not looked at the wireshark output good enough!
Unfortunately I cannot change it in our HttpProxy: this change of the case of 
the header keys is performed by the underlying HTTP Layer in java and not by 
our own code. I think the reason for this is that the HTTP specification states 
that the header-keys are case-insensitive (see f.i. 
http://stackoverflow.com/questions/5258977/are-http-headers-case-sensitive).
Maybe if this bug is fixed in the svn client then the segmentation fault would 
be solved? There might be another header field which is not managed correctly 
because of case-sensitiveness?

Remarque concernant la sécurité:
Ce courriel provenant de PostFinance est signé. Vous trouverez d'autres 
informations à ce sujet sous: 
https://www.postfinance.ch/e-signature.
Ne divulguez jamais vos éléments de sécurité à des tiers.

smime.p7s
Description: S/MIME Cryptographic Signature

AW: AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-09 Thread pierre.viret

> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Freitag, 9. Januar 2015 12:19
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to
> serf
> 
>  writes:
> 
> > Yes you are right, I have not looked at the wireshark output good
> > enough!  Unfortunately I cannot change it in our HttpProxy: this
> > change of the case of the header keys is performed by the underlying
> > HTTP Layer in java and not by our own code. I think the reason for
> > this is that the HTTP specification states that the header-keys are
> > case-insensitive (see f.i.
> > http://stackoverflow.com/questions/5258977/are-http-headers-case-sensitive).
> 
> It's fixed on trunk and proposed for the next release.
> 
> > Maybe if this bug is fixed in the svn client then the segmentation
> > fault would be solved? There might be another header field which is
> > not managed correctly because of case-sensitiveness?
> 
> I noticed that your proxy is changing the status line by dropping the
> reason-phrase:
> 
>HTTP/1.1 207 Multi-Status
> 
> to
> 
>HTTP/1.1 207
> 
> The client is allowed to ignore the reason-phrase but I don't know whether the
> server is allowed to omit it.  A strict reading of the RFC may require at 
> least a
> space after the 207.  However, this is not the cause of the crash, at least 
> on my
> machine.
> 
> I can't reproduce the crash.  I see the trace has "Connection: close" so the 
> problem
> is not probably not extra data between requests.  The trace is for a Windows 
> client,
> is it possible to get a trace for the Linux client?
> 
> I've converted the trace to a script.  Does your Windows client crash when run
> against the dummy server in this script?  Does the Linux client crash?

Good news!
I could create a new implementation of the JDK HttpServer which fixes the 207 
Status line problem, and I can't reproduce the segmentation fault anymore when 
the HttpProxy uses this fixed version.
This means that the SVN Client 1.8 now runs successfully the "ls" command if 
the HttpProxy sends a status line " HTTP/1.1 207 Multi-Status" correctly as 
stated in the specification. I have performed some tests using TortoiseSVN 1.8 
on windows, too, without any error.
We will perform further tests and will inform you in case we get again some 
error, but I'm optimist and confident that this solves the problem.

Question: do you know when the patch for the header field name 
case-sensitiveness will be delivered? Is it planned for 1.8.12 SVN client? For 
me it's important: if we have to wait for a too long time I would try to 
override the default behavior in Java to ensure that the case of the header 
field names get not modified and to avoid performance issues.

I think this is important to fix the snv code which causes the segmentation 
fault in case the status line is missing the space character: this would avoid 
client crashes with HTTP servers & proxies using the HttpServer implementation 
included in the Java Virtual Machine. I will post an issue by Java but I have 
no idea how quickly the problem with the 207 return code would be fix in there.

At this place I would like to thank you very much for your support which is a 
great help for us!

Here the output of socat on debian when I use the fixed HttpServer version: 
note the correct status line for code 207! In this case I could not reproduce 
the segmentation fault, I tried it on windows and on debian.

PROPFIND /svn/t_sponis_testrepo/!svn/bc/16 HTTP/1.1\r
Host: 127.0.0.1:9630\r
User-Agent: SVN/1.8.10 (x86_64-pc-linux-gnu) serf/1.3.7\r
Content-Type: text/xml\r
Accept-Encoding: gzip\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
Depth: 1\r
Transfer-Encoding: chunked\r
\r
71\r
\r
0\r
\r
< 2015/01/09 14:40:40.993585  length=474 from=0 to=473
HTTP/1.1 207 Multi-Status\r
Content-type: text/xml; charset="utf-8"\r
Content-length: 1052\r
Ch.nevis.session.secroles: 
auth.strong,idma_Benutzer,infplat_user,esclienttest_roleBC,esclienttest_roleA,esclienttest_roleB,depo_Admin,itam_Leser,itam_Administrator,itsms_asset_viewer,itsms_svc_req_usr,testsaml_admin,global.allow,acc.idma,acc.infplat,acc.esclienttest,acc.depo,acc.itam,acc.itsms,acc.testsaml\r
Connection: close\r
Server: Apache\r
Date: Fri, 09 Jan 2015 13:40:40 GMT\r
\r
< 2015/01/09 14:40:41.033014  length=1 from=474 to=474
<< 2015/01/09 14:40:41.033281  length=1051 from=475 to=1525
?xml version="1.0" encoding="utf-8"?>

/svn/t_sponis_testrepo/!svn/bc/16/

HTTP/1.1 200 OK

/svn/t_sponis_testrepo/!svn/bc/16/trunk/

HTTP/1.1 200 OK

/svn/t_sponis_testrepo/!svn/bc/16/branches/

HTTP/1.1 200 OK

/svn/t_sponis_testrepo/!svn/bc/16/tags/

HTTP/1.1 200 OK

Regards,
Pierre

Remarque concernant la sécurité:
Ce courriel provenant de PostFina

AW: AW: AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

2015-01-09 Thread pierre.viret



> -Ursprüngliche Nachricht-
> Von: Philip Martin [mailto:philip.mar...@wandisco.com]
> Gesendet: Freitag, 9. Januar 2015 16:25
> An: Viret Pierre, PF54
> Cc: users@subversion.apache.org
> Betreff: Re: AW: AW: AW: AW: AW: AW: Segmentation Fault with SVN Client
> related to serf
> 
>  writes:
> 
> > Question: do you know when the patch for the header field name
> > case-sensitiveness will be delivered? Is it planned for 1.8.12 SVN
> > client?
> 
> It's approved for 1.8.12, see:
> 
>   http://svn.apache.org/repos/asf/subversion/branches/1.8.x/STATUS
> 

I have implemented a small workaround which avoids that the case of all header 
field names string with "SVN-" get changed. This seems to work properly: I send 
you the output of socat.
Can you please confirm that the downgrading from v2 to v1 is away now?

I still have to analyse the "Connection: close" problem.

> 2015/01/09 16:35:40.817050  length=521 from=0 to=520
OPTIONS /svn/t_sponis_testrepo HTTP/1.1\r
Host: 127.0.0.1:9630\r
User-Agent: SVN/1.8.10 (x86_64-pc-linux-gnu) serf/1.3.7\r
Content-Type: text/xml\r
Connection: keep-alive\r
Accept-Encoding: gzip\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
Content-Length: 131\r
\r
<
 2015/01/09 16:35:41.030351  length=1861 from=0 to=1860
HTTP/1.1 200 OK\r
SVN-Repository-MergeInfo: yes\r
Content-type: text/xml; charset="utf-8"\r
SVN-Allow-Bulk-Updates: On\r
SVN-Txn-Root-Stub: /svn/t_sponis_testrepo/!svn/txr\r
SVN-Rev-Root-Stub: /svn/t_sponis_testrepo/!svn/rvr\r
SVN-VTxn-Root-Stub: /svn/t_sponis_testrepo/!svn/vtxr\r
Content-length: 201\r
Dav: 1,2, version-control,checkout,working-resource, 
merge,baseline,activity,version-controlled-collection, 
http://subversion.tigris.org/xmlns/dav/svn/depth, 
http://subversion.tigris.org/xmlns/dav/svn/log-revprops, 
http://subversion.tigris.org/xmlns/dav/svn/atomic-revprops, 
http://subversion.tigris.org/xmlns/dav/svn/partial-replay, 
http://subversion.tigris.org/xmlns/dav/svn/inherited-props, 
http://subversion.tigris.org/xmlns/dav/svn/inline-props, 
http://subversion.tigris.org/xmlns/dav/svn/reverse-file-revs, 
http://subversion.tigris.org/xmlns/dav/svn/mergeinfo, 
http://subversion.tigris.org/xmlns/dav/svn/ephemeral-txnprops, 
http://subversion.tigris.org/xmlns/dav/svn/replay-rev-resource\r
Ch.nevis.session.secroles: 
auth.strong,idma_Benutzer,infplat_user,esclienttest_roleBC,esclienttest_roleA,esclienttest_roleB,depo_Admin,itam_Leser,itam_Administrator,itsms_asset_viewer,itsms_svc_req_usr,testsaml_admin,global.allow,acc.idma,acc.infplat,acc.esclienttest,acc.depo,acc.itam,acc.itsms,acc.testsaml\r
SVN-Repository-UUID: 0926ec5e-c495-11e3-b81a-bb1aca739395\r
Ms-author-via: DAV\r
Connection: close\r
Server: Apache\r
SVN-VTxn-Stub: /svn/t_sponis_testrepo/!svn/vtxn\r
SVN-Rev-Stub: /svn/t_sponis_testrepo/!svn/rev\r
Date: Fri, 09 Jan 2015 15:35:41 GMT\r
SVN-Me-Resource: /svn/t_sponis_testrepo/!svn/me\r
SVN-Txn-Stub: /svn/t_sponis_testrepo/!svn/txn\r
Allow: 
OPTIONS,GET,HEAD,POST,DELETE,TRACE,PROPFIND,PROPPATCH,COPY,MOVE,LOCK,UNLOCK,CHECKOUT\r
SVN-Repository-Root: /svn/t_sponis_testrepo\r
SVN-Supported-Posts: create-txn, create-txn-with-props\r
SVN-Youngest-Rev: 16\r
\r
< 2015/01/09 16:35:41.069585  length=1 from=1861 to=1861
<< 2015/01/09 16:35:41.069780  length=200 from=1862 to=2061
?xml version="1.0" encoding="utf-8"?>

/svn/t_sponis_testrepo/!svn/act/
> 2015/01/09 16:35:41.172067  length=426 from=0 to=425
OPTIONS /svn/t_sponis_testrepo HTTP/1.1\r
Host: 127.0.0.1:9630\r
User-Agent: SVN/1.8.10 (x86_64-pc-linux-gnu) serf/1.3.7\r
Accept-Encoding: gzip\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/depth\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
DAV: http://subversion.tigris.org/xmlns/dav/svn/log-revprops\r
Transfer-Encoding: chunked\r
\r
42\r
\r
0\r
\r
< 2015/01/09 16:35:41.351560  length=1142 from=0 to=1141
HTTP/1.1 200 OK\r
Content-type: text/xml; charset="utf-8"\r
Content-length: 97\r
Ch.nevis.session.secroles: 
auth.strong,idma_Benutzer,infplat_user,esclienttest_roleBC,esclienttest_roleA,esclienttest_roleB,depo_Admin,itam_Leser,itam_Administrator,itsms_asset_viewer,itsms_svc_req_usr,testsaml_admin,global.allow,acc.idma,acc.infplat,acc.esclienttest,acc.depo,acc.itam,acc.itsms,acc.testsaml\r
Dav: 1,2, version-control,checkout,working-resource, 
merge,baseline,activity,version-controlled-collection, 
http://subversion.tigris.org/xmlns/dav/svn/depth, 
http://subversion.tigris.org/xmlns/dav/svn/log-revprops, 
http://subversion.tigris.org/xmlns/dav/svn/atomic-revprops, 
http://subversion.tigris.org/xmlns/dav/svn/partial-replay, 
http://subversion.tigris.org/xmlns/dav/svn/inherited-props, 
http://subversion.tigris.org/xmlns/dav/svn/inline-props, 
http://subversion.tigris.org/xmlns/dav/svn/reverse-file-revs, 
http://subversion.tigris.org/xmlns/dav/svn/mergeinfo\r
Ms-author-via: DAV\r
Connection: close\r
Server: Apache\r
Da

Segmentation Fault with SVN Client related to serf

AW: Segmentation Fault with SVN Client related to serf

AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

AW: AW: AW: AW: AW: AW: AW: Segmentation Fault with SVN Client related to serf

10 matches

Site Navigation

Mail list logo

Footer information