Hi!
Some NICs (e.g. bnxt) change their RSS indirection table size based on
the queue count, because the hardware table is a shared resource. The
ethtool core locks ctx->indir_size at context creation, so drivers
have to reject channel changes when RSS contexts exist.
This series adds resize helpers and wires them up in bnxt. It also
adds tracking of the user provided indirection table size to the
ethtool core.
Patch 1 tracks the user-provided indirection table size (user_size)
in ctx->indir_user_size for non-default RSS contexts and in
dev->ethtool->rss_indir_user_size for context 0. It is set when the
indirection table is configured via netlink or ioctl, and cleared to
zero on reset-to-default.
IFF_RXFH_CONFIGURED is removed, and replaced with rss_indir_user_size.
The flag is redundant now that user_size captures the same
information.
Patch 2 adds core resize helpers:
ethtool_rxfh_indir_can_resize() - read-only validation for context 0
ethtool_rxfh_indir_resize() - fold/unfold context 0 table in place
ethtool_rxfh_ctxs_can_resize() - validate all non-default contexts
ethtool_rxfh_ctxs_resize() - resize all non-default contexts,
with locking and RSS_NTF notifications
Patch 3 uses the resize helpers in bnxt_set_channels().
Patch 4 adds HW tests in rss_drv.py (devices without dynamic table
sizing are skipped):
resize_periodic - fold/unfold with a non-default [3,2,1,0]
sub-table (user_size=4), verifying exact content preservation
(main + ctx)
resize_below_user_size_reject - periodic sub-table with user_size
between big and small device table sizes; verifies that shrinking
below user_size is rejected even when the table is periodic
(main + ctx)
resize_nonperiodic_reject - non-periodic table blocks channel
reduction, with an extra periodic context to exercise
multi-context validation (main + ctx)
resize_nonperiodic_no_corruption - failed resize leaves table
contents and channel count unchanged (main + ctx)
Running the tests:
# On real hardware
sudo NETIF=eth0 ./rss_drv.py
Changes v6 -> v7:
- Renamed ethtool_rxfh_indir_clear() to ethtool_rxfh_indir_lost() to
align with ethtool_rxfh_context_lost() naming (Jakub)
- Updated kdoc to clearly state when the function should called (Jakub)
- Added netdev_err() inside ethtool_rxfh_indir_lost() similar to
ethtool_rxfh_context_lost() (Jakub)
- In mlx5 mlx5e_attach_netdev(), moved conditional rtnl_lock() above
the channel-reduction block so ethtool_rxfh_indir_lost() runs under
RTNL when the netdev is registered (needed for ethnl_notify())
(sashiko.dev)
Changes v5 -> v6:
- rss_set_prep_indir() returns user_size (>= 0) on success instead
of using a u32 *user_sizep output parameter (Jakub)
- Removed IFF_RXFH_CONFIGURED; netif_is_rxfh_configured() now checks
dev->ethtool->rss_indir_user_size instead of priv_flags (Jakub)
- Added ethtool_rxfh_indir_lost() helper for drivers that
forcibly reset indirection tables. Sends ETHTOOL_MSG_RSS_NTF
notification. Converted bnxt and mlx5 to use it (Jakub)
- ethtool_rxfh_indir_can_resize()/ethtool_rxfh_indir_resize() wrap
the internal resize logic for context 0, reading user_size from
dev->ethtool->rss_indir_user_size. Removes the need for
netif_is_rxfh_configured() guards in drivers (Jakub)
- Removed netif_is_rxfh_configured() guards in bnxt_set_channels()
and nsim_set_channels() (Jakub)
- More consistent function names
Changes v4 -> v5:
- Track user-provided indirection table size (user_size) as a resize
floor. Added indir_user_size to ethtool_rxfh_context and
rss_indir_user_size to ethtool_netdev_state. ethtool_rxfh_can_resize()
and ethtool_rxfh_resize() now take a user_size parameter and reject
shrinking below it. (Jakub)
- Propagated user_size out of rss_set_prep_indir() and stored it on
successful set in both netlink and ioctl paths.
- resize_periodic test now sends a 4-entry sub-table (user_size=4)
instead of replicating to full device table size
- Added resize_below_user_size_reject test to verify user_size floor.
- Removed "Open items" section - user_size tracking is now implemented.
Changes v3 -> v4:
- Rebased onto net-next
- Added Reviewed-by: from Michael
- Added missing Cc: to make the pwbots happier
Changes v2 -> v3:
- Changed ethtool_rxfh_can_resize() to return bool instead of int;
true means resize is possible, false means it is not. Inverted
callers accordingly. (Jakub)
- Added Tested-by from Pavan
Changes v1 -> v2:
- Dropped netdevsim support and netdevsim selftest (Jakub)
- Split ethtool_rxfh_contexts_resize_all() into separate validate
(ethtool_rxfh_ctxs_can_resize) and apply (ethtool_rxfh_ctxs_resize)
so drivers can validate before closing the device (Jakub)
- Shortened helper names (Jakub)
- Replaced scoped_guard(mutex) with explicit mutex_lock/unlock
(Jakub)
- Removed defensive zero-size check, bare expressions instead of != 0
comparisons, ! instead of == 0 (Jakub)
- In bnxt, moved bnxt_check_rings() before RSS validation and
deferred actual resize to after bnxt_close_nic() (Jakub, Michael)
- Added comment that RSS table size only changes on P5 chips with
older firmware (Michael)
- Use non-default [3,2,1,0]xN pattern set via netlink to distinguish
correct fold from driver resetting to defaults (Jakub)
- Check exact indirection table pattern, not just set(indir) (Jakub)
- Use ksft_raises() instead of try/except/else (Jakub)
- Removed queue_count=8 from NetDrvEnv (Jakub)
- Added ksft_variants to resize_nonperiodic_reject for ctx coverage
- Added extra periodic context in reject test for multi-context
validation coverage
- Added resize_nonperiodic_no_corruption test
Björn Töpel (4):
ethtool: Track user-provided RSS indirection table size
ethtool: Add RSS indirection table resize helpers
bnxt_en: Resize RSS contexts on channel count change
selftests: rss_drv: Add RSS indirection table resize tests
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 3 +-
.../net/ethernet/broadcom/bnxt/bnxt_ethtool.c | 35 ++-
.../net/ethernet/mellanox/mlx5/core/en_main.c | 21 +-
include/linux/ethtool.h | 13 +
include/linux/netdevice.h | 7 +-
net/ethtool/common.c | 183 ++++++++++++++
net/ethtool/ioctl.c | 9 +-
net/ethtool/rss.c | 24 +-
.../selftests/drivers/net/hw/rss_drv.py | 233 +++++++++++++++++-
9 files changed, 488 insertions(+), 40 deletions(-)
base-commit: ca7e99335aea7c5977683624ba319157a4603f96
--
2.53.0
