On Fri, Sep 13, 2019 at 3:37 PM Thomas Higdon <t...@fb.com> wrote:
>
> For receive-heavy cases on the server-side, we want to track the
> connection quality for individual client IPs. This counter, similar to
> the existing system-wide TCPOFOQueue counter in /proc/net/netstat,
> tracks out-of-order packet reception. By providing this counter in
> TCP_INFO, it will allow understanding to what degree receive-heavy
> sockets are experiencing out-of-order delivery and packet drops
> indicating congestion.
>
> Please note that this is similar to the counter in NetBSD TCP_INFO, and
> has the same name.
>
> Signed-off-by: Thomas Higdon <t...@fb.com>
> ---
>
> no changes from v3
>
> include/linux/tcp.h | 2 ++
> include/uapi/linux/tcp.h | 2 ++
> net/ipv4/tcp.c | 2 ++
> net/ipv4/tcp_input.c | 1 +
> 4 files changed, 7 insertions(+)
>
> diff --git a/include/linux/tcp.h b/include/linux/tcp.h
> index f3a85a7fb4b1..a01dc78218f1 100644
> --- a/include/linux/tcp.h
> +++ b/include/linux/tcp.h
> @@ -393,6 +393,8 @@ struct tcp_sock {
> */
> struct request_sock *fastopen_rsk;
> u32 *saved_syn;
> +
> + u32 rcv_ooopack; /* Received out-of-order packets, for tcpinfo */
Thanks for adding this.
A thought: putting the new rcv_ooopack field here makes struct
tcp_sock bigger, and increases the odds of taking a cache miss
(according to "pahole" this field is the only one in a new cache
line).
I'd suggest putting the new rcv_ooopack field immediately before
rcv_rtt_last_tsecr. That would use up a 4-byte hole, and would place
it in a cache line already used on TCP receivers (for rcv_rtt logic).
This would make it less likely this new field causes more cache misses
or uses more space.
Details: looking at the output of "pahole" for tcp_sock in various cases:
net-next before this patch:
-------------------------------------
...
u8 bpf_sock_ops_cb_flags; /* 2076 1 */
/* XXX 3 bytes hole, try to pack */
u32 rcv_rtt_last_tsecr; /* 2080 4 */
/* XXX 4 bytes hole, try to pack */
struct {
u32 rtt_us; /* 2088 4 */
u32 seq; /* 2092 4 */
u64 time; /* 2096 8 */
} rcv_rtt_est; /* 2088 16 */
...
/* size: 2176, cachelines: 34, members: 134 */
/* sum members: 2164, holes: 4, sum holes: 12 */
/* paddings: 3, sum paddings: 12 */
net-next with this patch:
-------------------------------------
...
u32 * saved_syn; /* 2168 8 */
/* --- cacheline 34 boundary (2176 bytes) --- */
u32 rcv_ooopack; /* 2176 4 */
...
/* size: 2184, cachelines: 35, members: 135 */
/* sum members: 2168, holes: 4, sum holes: 12 */
/* padding: 4 */
/* paddings: 3, sum paddings: 12 */
/* last cacheline: 8 bytes */
net-next with this field in the suggested spot:
-------------------------------------
...
/* XXX 3 bytes hole, try to pack */
u32 rcv_ooopack; /* 2080 4 */
u32 rcv_rtt_last_tsecr; /* 2084 4 */
struct {
u32 rtt_us; /* 2088 4 */
u32 seq; /* 2092 4 */
u64 time; /* 2096 8 */
} rcv_rtt_est; /* 2088 16 */
...
/* size: 2176, cachelines: 34, members: 135 */
/* sum members: 2168, holes: 3, sum holes: 8 */
/* paddings: 3, sum paddings: 12 */
neal
neal
