On Mon, Feb 08, 2021 at 07:03:59PM +0100, Jan Klemkow wrote: > On Mon, Feb 08, 2021 at 03:42:54PM +0100, Alexander Bluhm wrote: > > On Wed, Feb 03, 2021 at 11:20:04AM +0100, Claudio Jeker wrote: > > > Just commit it. OK claudio@ > > > If people see problems we can back it out again. > > > > This has huge impact on TCP performance. > > > > http://bluhm.genua.de/perform/results/2021-02-07T00%3A01%3A40Z/perform.html > > > > For a single TCP connection between to OpenBSD boxes, througput > > drops by 77% from 3.1 GBit/sec to 710 MBit/sec. But with 100 > > parallel connections the througput over all increases by 5%. > > For single connections our kernel is limited to send out 4 max TCP > segments. I don't see that, because I just measured with 10 and 30 > streams in parallel. > > FreeBSD disabled it 20 yeas ago. > https://github.com/freebsd/freebsd-src/commit/d912c694ee00de5ea0f46743295a0fc603cab562
TCP_MAXBURST was added together with SACK in rev 1.12 of tcp_output.c to our code base. ---------------------------- revision 1.12 date: 1998/11/17 19:23:02; author: provos; state: Exp; lines: +239 -14; NewReno, SACK and FACK support for TCP, adapted from code for BSDI by Hari Balakrishnan (h...@lcs.mit.edu), Tom Henderson (t...@cs.berkeley.edu) and Venkat Padmanabhan (padma...@cs.berkeley.edu) as part of the Daedalus research group at the University of California, (http://daedalus.cs.berkeley.edu). [I was able to do this on time spent at the Center for Information Technology Integration (citi.umich.edu)] ---------------------------- > I would suggest to remove the whole feature. Sending 4 segments per call to tcp_output() cannot scale. Bandwith increases, window size grows, but segment size is 1500 for decades. With this diff on top of jan's delay ACK behavior I get 4.1 GBit/sec over a single TCP connection using tcpbench -S1000000. Before both changes it was only 3.0. I recommend removing TCP_MAXBURST like FreeBSD did. bluhm > Index: tcp.h > =================================================================== > RCS file: /cvs/src/sys/netinet/tcp.h,v > retrieving revision 1.21 > diff -u -p -r1.21 tcp.h > --- tcp.h 10 Jul 2019 18:45:31 -0000 1.21 > +++ tcp.h 8 Feb 2021 17:52:38 -0000 > @@ -105,8 +105,6 @@ struct tcphdr { > #define TCP_MAX_SACK 3 /* Max # SACKs sent in any segment */ > #define TCP_SACKHOLE_LIMIT 128 /* Max # SACK holes per connection */ > > -#define TCP_MAXBURST 4 /* Max # packets after leaving Fast > Rxmit */ > - > /* > * Default maximum segment size for TCP. > * With an IP MSS of 576, this is 536, > Index: tcp_output.c > =================================================================== > RCS file: /cvs/src/sys/netinet/tcp_output.c,v > retrieving revision 1.129 > diff -u -p -r1.129 tcp_output.c > --- tcp_output.c 25 Jan 2021 03:40:46 -0000 1.129 > +++ tcp_output.c 8 Feb 2021 17:53:07 -0000 > @@ -203,7 +203,6 @@ tcp_output(struct tcpcb *tp) > int idle, sendalot = 0; > int i, sack_rxmit = 0; > struct sackhole *p; > - int maxburst = TCP_MAXBURST; > #ifdef TCP_SIGNATURE > unsigned int sigoff; > #endif /* TCP_SIGNATURE */ > @@ -1120,7 +1119,7 @@ out: > tp->last_ack_sent = tp->rcv_nxt; > tp->t_flags &= ~TF_ACKNOW; > TCP_TIMER_DISARM(tp, TCPT_DELACK); > - if (sendalot && --maxburst) > + if (sendalot) > goto again; > return (0); > }
