The MANA driver can fail to bring up its queues on systems with high
memory utilization because every GDMA queue ring is allocated as a
single dma_alloc_coherent() of the whole power-of-2 ring size. Under
memory fragmentation these high-order allocations may fail, preventing
the driver from creating queues when opening the interface, after a VF
reset, or when reconfiguring channels, ring parameters or MTU.
Per-queue sizes that are problematic, with depth and size given as
(default, max) over the ethtool ring settings:
ring entry depth size
------------------------------------------------------------
TX completion queue 64 B (256, 16384) (16 KB, 1024 KB)
TX send queue 32 B (256, 16384) ( 8 KB, 512 KB)
RX completion queue 64 B (1024, 8192) (64 KB, 512 KB)
RX receive queue 32 B (1024, 8192) (32 KB, 256 KB)
event queue 16 B 2048 (fixed) 32 KB
This series addresses the issue by:
1. Routing all CPU-side ring access through mana_gd_ring_ptr() and
mana_gd_ring_contig_avail(). On a contiguous ring these reduce to
simple arithmetic, so this patch is a pure refactor.
2. Falling back in mana_gd_alloc_memory() to a vector of scattered
order-0 coherent pages when the contiguous allocation fails. The
device sees the same page-list format either way, as
mana_gd_create_dma_region() already describes a ring as a list of
MANA_PAGE_SIZE addresses. The HW channel stays contiguous, as
advertising a scattered page list needs the HW channel itself.
Throughput testing confirms no regression. Since the fallback only
triggers under memory fragmentation, the scattered-page path was enabled
unconditionally for all eligible GDMA queue rings during testing (iperf3,
Gbit/s):
Baseline Patched Patched
Connections Contiguous Contiguous Scattered
-----------------------------------------------
1 46.1 46.2 46.1
16 182 182 182
32 182 182 182
64 182 182 182
Aditya Garg (2):
net: mana: Route ring-buffer access through offset-based helpers
net: mana: Fall back to scattered pages for GDMA queues
.../net/ethernet/microsoft/mana/gdma_main.c | 219 +++++++++++++++---
.../net/ethernet/microsoft/mana/hw_channel.c | 2 +-
drivers/net/ethernet/microsoft/mana/mana_en.c | 3 +
include/net/mana/gdma.h | 19 +-
4 files changed, 205 insertions(+), 38 deletions(-)
--
2.43.0
