> -----Original Message-----
> From: Hamza Mahfooz <[email protected]>
> Sent: Tuesday, October 13, 2026 5:50 PM
> To: [email protected]
> Cc: Haiyang Zhang <[email protected]>; Wei Liu <[email protected]>;
> Dexuan Cui <[email protected]>; Andrew Lunn <[email protected]>;
> David S. Miller <[email protected]>; Eric Dumazet <[email protected]>;
> Jakub Kicinski <[email protected]>; Paolo Abeni <[email protected]>;
> Konstantin Taranov <[email protected]>; Simon Horman
> <[email protected]>; Alexei Starovoitov <[email protected]>; Daniel Borkmann
> <[email protected]>; Jesper Dangaard Brouer <[email protected]>; John
> Fastabend <[email protected]>; Stanislav Fomichev
> <[email protected]>; [email protected]; linux-
> [email protected]; [email protected]; [email protected];
> Hamza Mahfooz <[email protected]>
> Subject: [PATCH net-next v3] net: mana: add BQL support
>
> MANA doesn't support byte queue limits, so each TX queue can hold 256
> WQEs by default, and up to 16384, of up to 64 KB TSO packets, before the
> qdisc layer sees any backpressure. This adds latency to all the flows
> sharing a queue, and defeats AQM qdiscs such as fq_codel, which rely on
> BQL to keep the driver queues short.
>
> Report the bytes queued and completed to BQL, except for the XDP_TX and
> XDP_REDIRECT frames. These share the SQs, but bypass the qdisc and are
> sent even when the queue is stopped, so BQL can't hold them back.
> Counting them would let sustained XDP traffic keep the queue stopped,
> starving the stack, and the TX watchdog would then either fire while the
> completions still arrive, or, if the XDP sends refreshed trans_start,
> not fire when the completions stop. The XDP SKBs are marked in their
> skb->cb, next to the WQE size, so that their completions are skipped as
> well.
>
> Reset the BQL state when destroying the queues, once their NAPI is
> disabled, since the SKBs still pending after a drain timeout are freed
> without being completed.
>
> Lost TX completions can now stop a queue sooner: it also stops when the
> bytes in flight exceed the BQL limit, not only when the SQ is full, and
> the limit may never grow enough to cover the lost bytes. This includes
> the completions skipped by the WARN_ON_ONCE() early returns in
> mana_poll_tx_cq(), which already leak SQ space: completing the SKBs
> already freed there wouldn't reliably balance BQL, since
> mana_gd_poll_cq() has already consumed the CQEs of the rest of the
> batch. The TX watchdog then fires, as nothing refreshes the trans_start
> of a queue stopped by BQL, and its handler resets the queues along with
> their BQL state. The exception, as before this patch, is XDP traffic
> keeping the SQ full: each XDP send then stops the queue again,
> refreshing its trans_start.
>
> Tested on two Standard_D16ds_v6 Azure VMs in a proximity placement
> group, where the throughput is capped at 10.6 Gbps, with the DUT
> alternately booting net-next (c1c1f0a31712) with and without this patch
> applied.
>
> The throughput stays at the cap with 1, 8 and 64 TCP flows, and the idle
> TCP_RR latency doesn't change (p50 of 47 us with vs. without the patch).
> With 16 senders of 64-byte UDP packets, the DUT sends 3.7 to 4.0 Mpps
> without the patch and 3.7 to 4.1 Mpps with it. With bulk TCP flows
> sharing the queues, the netperf TCP_RR latency drops (medians of 6
> runs):
>
> 16 queues, 64 flows: p50 4649 -> 2182 us, p99 5621 -> 2677 us
> 1 queue, 8 flows: p50 2554 -> 418 us, p99 2723 -> 624 us
>
> The standing queue moves from the SQs to fq_codel, which now drops
> packets: about 2500 qdisc drops and 88K TCP retransmissions per 48 s run
> with 64 flows, against at most 29 retransmissions before. The
> retransmissions are about 0.2% of the segments sent. The TCP_RR flows
> lose no packets, and the packets of the bulk flows are dropped before
> they reach the wire, so the goodput changes by less than 0.01%.
>
> Signed-off-by: Hamza Mahfooz <[email protected]>
Thank you!
Reviewed-by: Haiyang Zhang <[email protected]>
