On 09.01.2014 20:19, Branko Čibej wrote: > On 09.01.2014 17:09, Mojca Miklavec wrote: >> I'm unable to reproduce the faulty behaviour if I do a checkout from >> the same network where the server is located, no matter what I try >> (upgrading SVN client doesn't "help" triggering the error). Philip >> also said that he had no problem doing a checkout with client version >> 1.8.5 or 1.7. > > This confirms my suspicion that the error is triggered by some part of > the network infrastructure between your server and the outside world. > That's why I asked if there is a load-balancer involved. It could also > be caused by some kind of transparent proxy, or even a packet > analyzer. I doubt that your server is open to the world without some > kind of security measures in place. > > To be clear, I'm not saying that any of these things are configured > incorrectly; only that they may be interacting with Subversion in a > way that we don't handle well. One of the major differences between > 1.7 (which works) and 1.8 (which fails) is that we try to work around > issues with non-standard behaviour of certain "transparent" (sic) > proxies; and we can't claim to have covered all the possibilities. > > I can't see a way to figure out what's going on without help from your > network admins; we need some insight into why the connection is being > reset on the server side, and analyzing the TCP stream on the client > can't tell us that. > > > BTW, if you think it'd help to try a live debugging session, I'm only > about an hour's drive away from IJS.
So to wrap this up: they managed to fix the problem themselves, and it was indeed "some part of the network infrastructure;" the specifics are as follows: They have a Cisco ASA 5580 running IOS 9.1(4), and they had HTTP protocol inspection turned on; the configuration was as follows: policy-map type inspect http HTTP-CONTROL parameters protocol-violation action log policy-map global_policy class inspection_default inspect http HTTP-CONTROL The ASA was closing the connections, and their logs contained one of the following two reasons: %ASA-4-415016: policy-map HTTP-CONTROL:Maximum number of unanswered HTTP requests exceeded - Resetting connection from Ext:x.x.x.x/59769 to Int:y.y.y.y/80 %ASA-4-507003: tcp flow from Ext:x.x.x.x/59769 to Int:y.y.y.y/80 terminated by inspection engine, reason - reset unconditionally. The only reasonable explanation we could come up with was that moderately low bandwidth and high latency between client and server, coupled with the fact that some of the files in the repository are rather large and take a while to transfer, caused the 1.8 client to queue up enough pipelined GET requests during checkout that the firewall decided to call quits. A 1.7 client (using serf) did not exhibit this problem, because it also sends PROPGETS, and this apparently changed the timing enough that the number of pipelined requests never exceeded the ASA's configured maximum. Apparently this is not a new problem, having been reported before: https://supportforums.cisco.com/thread/2088590 They fixed the issue by switching off HTTP protocol inspection on the ASA. Interestingly enough, this also fixed a number of intermittent issues with plain ol' Web browsing that they had on occasion, so this is not specific to Subversion (as the link above also suggests), but is rather a bug^Wserious limitation of the ASA and/or IOS. -- Brane -- Branko Čibej | Director of Subversion WANdisco // Non-Stop Data e. br...@wandisco.com
