Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-08-14 Thread Marcelo Ricardo Leitner
On Fri, Aug 02, 2019 at 09:58:12PM +0200, Bernd wrote: > 2019-08-02 21:14 GMT+02:00, Neal Cardwell : > > What's the exact kernel version you are using? > > It is the RHEL errata kernel 3.10.0-957.21.3.el7 (rhsa-2019:1481), i > need to check if there is a newer one. FWIW, this one doesn't have the

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-08-02 Thread Bernd
2019-08-02 21:14 GMT+02:00, Neal Cardwell : > What's the exact kernel version you are using? It is the RHEL errata kernel 3.10.0-957.21.3.el7 (rhsa-2019:1481), i need to check if there is a newer one. > Eric submitted a patch recently that may address your issue: >tcp: be more careful in tcp_

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-08-02 Thread Neal Cardwell
On Fri, Aug 2, 2019 at 3:03 PM Bernd wrote: > > Hello, > > While analyzing a aborted upload packet capture I came across a odd > trace where a sender was not responding to a duplicate SACK but > sending further segments until it stalled. > > Took me some time until I remembered this fix, and actua

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-08-02 Thread Bernd
Hello, While analyzing a aborted upload packet capture I came across a odd trace where a sender was not responding to a duplicate SACK but sending further segments until it stalled. Took me some time until I remembered this fix, and actually the problems started since the security fix was applied

RE: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-16 Thread Prout, Andrew - LLSC - MITLL
On 7/11/2019 11:15 PM, Prout, Andrew - LLSC - MITLL wrote: > I in no way intended to imply that I had confirmed the small SO_SNDBUF was a > prerequisite to our problem. With my synthetic test, I was looking for a > concise reproducer and trying things to > stress the system. I've looked into th

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-12 Thread Eric Dumazet
On 7/11/19 9:04 PM, Jonathan Lemon wrote: > I discovered we have some production services that set SO_SNDBUF to > very small values (~4k), as they are essentially doing interactive > communications, not bulk transfers.  But there's a difference between > "terrible performance" and "TCP stops w

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Jonathan Lemon
On 11 Jul 2019, at 11:28, Eric Dumazet wrote: On 7/11/19 7:14 PM, Prout, Andrew - LLSC - MITLL wrote: In my opinion, if a small SO_SNDBUF below a certain value is no longer supported, then SOCK_MIN_SNDBUF should be adjusted to reflect this. The RCVBUF/SNDBUF sizes are supposed to be hints

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Eric Dumazet
On 7/11/19 8:26 PM, Michal Kubecek wrote: > > I'm aware it's not a realistic test. It was written as quick and simple > check of the pre-4.19 patch, but it shows that even TLP may not get > through. Most of TLP probes send new data, not rtx. But yes, I get your point. SO_SNDBUF=15000 in yo

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Eric Dumazet
On 7/11/19 7:14 PM, Prout, Andrew - LLSC - MITLL wrote: > > In my opinion, if a small SO_SNDBUF below a certain value is no longer > supported, then SOCK_MIN_SNDBUF should be adjusted to reflect this. The > RCVBUF/SNDBUF sizes are supposed to be hints, no error is returned if they > are not

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Michal Kubecek
On Thu, Jul 11, 2019 at 11:19:45AM +0200, Eric Dumazet wrote: > > > On 7/11/19 9:28 AM, Christoph Paasch wrote: > > > > > >> On Jul 10, 2019, at 9:26 PM, Eric Dumazet wrote: > >> > >> > >> > >> On 7/10/19 8:53 PM, Prout, Andrew - LLSC - MITLL wrote: > >>> > >>> Our initial rollout was v4.14.13

RE: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Prout, Andrew - LLSC - MITLL
On 7/10/19 3:27 PM, Eric Dumazet wrote: > On 7/10/19 8:53 PM, Prout, Andrew - LLSC - MITLL wrote: >> >> Our initial rollout was v4.14.130, but I reproduced it with v4.14.132 as >> well, reliably for the samba test and once (not reliably) with synthetic >> test I was trying. A patched v4.14.132

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Eric Dumazet
On 7/11/19 9:28 AM, Christoph Paasch wrote: > > Would it make sense to always allow the alloc in tcp_fragment when coming > from __tcp_retransmit_skb() through the retransmit-timer ? > > AFAICS, the crasher was when an attacker sends "fake" SACK-blocks. Thus, we > would still be protected f

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Eric Dumazet
On 7/11/19 9:28 AM, Christoph Paasch wrote: > > >> On Jul 10, 2019, at 9:26 PM, Eric Dumazet wrote: >> >> >> >> On 7/10/19 8:53 PM, Prout, Andrew - LLSC - MITLL wrote: >>> >>> Our initial rollout was v4.14.130, but I reproduced it with v4.14.132 as >>> well, reliably for the samba test and o

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-11 Thread Christoph Paasch
> On Jul 10, 2019, at 9:26 PM, Eric Dumazet wrote: > > > > On 7/10/19 8:53 PM, Prout, Andrew - LLSC - MITLL wrote: >> >> Our initial rollout was v4.14.130, but I reproduced it with v4.14.132 as >> well, reliably for the samba test and once (not reliably) with synthetic >> test I was tryin

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-10 Thread Eric Dumazet
On 7/10/19 8:53 PM, Prout, Andrew - LLSC - MITLL wrote: > > Our initial rollout was v4.14.130, but I reproduced it with v4.14.132 as > well, reliably for the samba test and once (not reliably) with synthetic test > I was trying. A patched v4.14.132 with this patch partially reverted (just >

RE: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-10 Thread Prout, Andrew - LLSC - MITLL
On 7/10/19 2:29 PM, Eric Dumazet wrote: > On 7/10/19 8:23 PM, Prout, Andrew - LLSC - MITLL wrote: >> On 6/17/19 8:19 PM, Christoph Paasch wrote: >>> >>> Yes, this does the trick for my packetdrill-test. >>> >>> I wonder, is there a way we could end up in a situation where we can't >>> retransmit a

RE: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-10 Thread Prout, Andrew - LLSC - MITLL
On 6/17/19 8:19 PM, Christoph Paasch wrote: > > Yes, this does the trick for my packetdrill-test. > > I wonder, is there a way we could end up in a situation where we can't > retransmit anymore? > For example, sk_wmem_queued has grown so much that the new test fails. > Then, if we legitimately ne

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-07-10 Thread Eric Dumazet
On 7/10/19 8:23 PM, Prout, Andrew - LLSC - MITLL wrote: > On 6/17/19 8:19 PM, Christoph Paasch wrote: >> >> Yes, this does the trick for my packetdrill-test. >> >> I wonder, is there a way we could end up in a situation where we can't >> retransmit anymore? >> For example, sk_wmem_queued has gro

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Eric Dumazet
On 6/17/19 8:53 PM, Christoph Paasch wrote: > On Mon, Jun 17, 2019 at 8:44 PM Eric Dumazet wrote: >> >> >> >> On 6/17/19 8:19 PM, Christoph Paasch wrote: >>> >>> Yes, this does the trick for my packetdrill-test. >>> >>> I wonder, is there a way we could end up in a situation where we can't >>>

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Christoph Paasch
On Mon, Jun 17, 2019 at 8:44 PM Eric Dumazet wrote: > > > > On 6/17/19 8:19 PM, Christoph Paasch wrote: > > > > Yes, this does the trick for my packetdrill-test. > > > > I wonder, is there a way we could end up in a situation where we can't > > retransmit anymore? > > For example, sk_wmem_queued h

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Eric Dumazet
On 6/17/19 8:19 PM, Christoph Paasch wrote: > > Yes, this does the trick for my packetdrill-test. > > I wonder, is there a way we could end up in a situation where we can't > retransmit anymore? > For example, sk_wmem_queued has grown so much that the new test fails. > Then, if we legitimately

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Christoph Paasch
On Mon, Jun 17, 2019 at 7:28 PM Eric Dumazet wrote: > > > > On 6/17/19 5:18 PM, Christoph Paasch wrote: > > > > Hi Eric, I now have a packetdrill test that started failing (see > > below). Admittedly, a bit weird test with the SO_SNDBUF forced so low. > > > > Nevertheless, previously this test wou

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Eric Dumazet
On 6/17/19 5:18 PM, Christoph Paasch wrote: > > Hi Eric, I now have a packetdrill test that started failing (see > below). Admittedly, a bit weird test with the SO_SNDBUF forced so low. > > Nevertheless, previously this test would pass, now it stalls after the > write() because tcp_fragment()

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Christoph Paasch
On Mon, Jun 17, 2019 at 10:05 AM Eric Dumazet wrote: > > Jonathan Looney reported that a malicious peer can force a sender > to fragment its retransmit queue into tiny skbs, inflating memory > usage and/or overflow 32bit counters. > > TCP allows an application to queue up to sk_sndbuf bytes, > so

Re: [PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Jonathan Lemon
On 17 Jun 2019, at 10:03, Eric Dumazet wrote: > Jonathan Looney reported that a malicious peer can force a sender > to fragment its retransmit queue into tiny skbs, inflating memory > usage and/or overflow 32bit counters. > > TCP allows an application to queue up to sk_sndbuf bytes, > so we nee

[PATCH net 2/4] tcp: tcp_fragment() should apply sane memory limits

2019-06-17 Thread Eric Dumazet
Jonathan Looney reported that a malicious peer can force a sender to fragment its retransmit queue into tiny skbs, inflating memory usage and/or overflow 32bit counters. TCP allows an application to queue up to sk_sndbuf bytes, so we need to give some allowance for non malicious splitting of retra