Re: Slow VM, svn client drops connection with FIN packet

[stuartm . coding]

On Tuesday, December 17, 2013 10:32:34 AM UTC-5, Branko Čibej wrote:
>
> On 17.12.2013 16:19, Stuart MacDonald wrote: 
> > The Wireshark trace shows clearly and conclusively this is a bug in 
> > the client. The client drops the connection by sending a FIN packet 
> > unexpectedly. 
>
> It could be waiting for a response from the proxy that never reaches the 
> client, and dropping the connection after a timeout. 
>
> Regardless, you should still try to reproduce this with the latest 
> client (ideally, both latest 1.7 and 1.8) to determine if the problem 
> has been fixed. Especially in 1.8, there have been a lot of improvements 
> in the HTTP protocol driver.
>

I'm using WANdisco's packages to test with, as it turned out to be easier 
to get those set up than to compile from source with all the optional bits 
I need.

I have seen the exact same issue on 1.7.14:
svn: E175002: REPORT of '/svn/repos//!svn/me': Could not read 
response body: connection was closed by server (http://)
and the Wireshark capture shows the client sending a [FIN, ACK] to close 
the connection, despite it only being a fraction of a second since the last 
server traffic. This always occurs just after the TCP window opening up 
again, not sure if that's significant.

This *also* occurs on 1.8.5, although the error message is more informative:
svn: E120106: ra_serf: The server sent a truncated HTTP response body.
The Wireshark capture is *exactly* *identical*; after some server traffic, 
the TCP window opens up and the client sends a [FIN, ACK] to close the 
connection.

Having taken a closer look at the network captures; the failure pattern is 
always:
- 1380 byte packets from the server
- an ACK from the client
- a TCP window update from the client (the window hadn't gone to 0 in any 
cases, this is just a regular update)
- in one case there's another window update from the client
- a delay of 0 - 4 seconds where there is no traffic in either direction
- the client sends the [FIN, ACK]

Now, I'm not sure what constitutes a truncated HTTP response body, as the 
traffic looks the same in both cases, large blocks of encoded data. So I 
think the 1.8.x error message is also incorrect, but possibly less 
incorrect than the 1.7.x error message.

What's the next step?

...Stu



 

Re: Slow VM, svn client drops connection with FIN packet

[stuartm . coding]

I replied yesterday, but my post seems to have disappeared.

I figured it out, and the key was your reproduction and my realisation I 
wasn't seeing a FIN from my server. It turns out that in every case the last
392598 71.906863000 10.222.3.88 10.14.11.50 HTTP 1434 Continuation or 
non-HTTP traffic
line from the server is actually also a FIN packet, and Wireshark (or the 
TCP dissector) is "hiding" that; if I look at the packet I can see the FIN 
flag, but it's not mentioned in the Info field in the packet list. So the 
FINs from my client are close handshake responses not initiations, and it 
is the server timing out after all.

Thanks for the help. I'm checking with my IT dept to see if they can bump 
up the timeout.

...Stu

On Tuesday, January 14, 2014 11:55:39 AM UTC-5, Philip Martin wrote:
>
> Stuart MacDonald > writes: 
>
> > svn: E175002: REPORT of '/svn/repos/deckard-65x/!svn/me': Could not read 
> > response body: connection was closed by server (http://foundry51.qnx.com) 
>
>
> I can provoke this on my local LAN by setting the Apache timeout on the 
> server to a low value such as 5 seconds.  Then I run a checkout that 
> generates a large REPORT response and I use gdb to interrupt the client 
> at subversion/libsvn_ra_neon/fetch.c:1709 so that the client is in the 
> middle of reading the repsonse.  I wait for the server timeout to 
> expire and then resume the client and I get the error: 
>
> svn: E175002: REPORT of '/obj/repo/!svn/me': Could not read response body: 
> connection was closed by server (http://peri.local:) 
>
> One fix is to increase the server timeout to handle your slow client. 
>
> -- 
> Philip Martin | Subversion Committer 
> WANdisco // *Non-Stop Data* 
>