The RSS key only extracted the outer IP header. Tunnelled traffic whose
outer headers are fixed then carries no entropy for the hash, so every
flow lands on a single Rx queue.
Extract both the outer IP (header index 0) and the innermost IP instance
(HDR_INDEX_LAST). Plain frames keep being hashed on their only IP header;
the inner extract resolves to nothing and adds no entropy. Tunnelled
frames are also hashed on their inner IP and spread across the Rx queues.
That a lone IP header does not resolve HDR_INDEX_LAST is the regression
reverted by commit 2b375df07e48 ("net/dpaa2: revert inner RSS level
support"): extracted only at HDR_INDEX_LAST, every plain frame hashed to
the same value and landed on one Rx queue.
The worst case, requesting every type the PMD advertises, is 16 extracts
out of the DPKG_MAX_NUM_OF_EXTRACTS 20. On LX2160A
dpni_set_rx_hash_dist() accepts the key, and plain IPv4, plain IPv6, VLAN
tagged and tunnelled IPv6 traffic all spread across the Rx queues.
This is the PMD default hash: dpaa2 does not expose the ethdev RSS level
selector, so it applies to every RSS request. The hardware cannot hash
the inner IP alone, as HDR_INDEX_LAST only resolves when several IP
headers are stacked and a plain frame would hash to a constant. Folding
the outer IP into the key is therefore unavoidable, and two tunnelled
flows that share an inner IP but differ in their outer IP may hash to
different queues. This is documented in the dpaa2 guide.
Signed-off-by: Maxime Leroy <[email protected]>
---
v3:
* Drop the DPAA2_DIST_HDR_INDEX_LAST define and take (uint8_t)LAST_HDR_INDEX
directly at its single use site, rather than adding a second name for a
constant the driver already has (Stephen Hemminger).
* Record the extract budget and the LX2160A results in the commit log
(Stephen Hemminger).
* Re-added the release notes entry dropped in v2, now under 26.11.
v2:
* Sent standalone: patch 1/2 of v1, the revert of the inner RSS level
support, was applied as 2b375df07e48.
* Dropped the release_26_07.rst entry, that release having been tagged.
Unchanged otherwise.
doc/guides/nics/dpaa2.rst | 3 ++
doc/guides/rel_notes/release_26_11.rst | 5 +++
drivers/net/dpaa2/base/dpaa2_hw_dpni.c | 60 +++++++++++++-------------
3 files changed, 37 insertions(+), 31 deletions(-)
diff --git a/doc/guides/nics/dpaa2.rst b/doc/guides/nics/dpaa2.rst
index ae8b32af2c..aaaf5f9713 100644
--- a/doc/guides/nics/dpaa2.rst
+++ b/doc/guides/nics/dpaa2.rst
@@ -588,6 +588,9 @@ Other Limitations
- RSS hash key cannot be modified.
- RSS RETA cannot be configured.
+- RSS hashes on both the outer and the inner IP header. Tunnelled flows
+ that share the same inner IP but differ in their outer IP may therefore
+ be steered to different Rx queues.
.. _dpaa2_dptmapi:
diff --git a/doc/guides/rel_notes/release_26_11.rst
b/doc/guides/rel_notes/release_26_11.rst
index 87c7e81bde..4887bccbc0 100644
--- a/doc/guides/rel_notes/release_26_11.rst
+++ b/doc/guides/rel_notes/release_26_11.rst
@@ -55,6 +55,11 @@ New Features
Also, make sure to start the actual text at the margin.
=======================================================
+* **Updated NXP dpaa2 driver.**
+
+ * Added the inner IP header to the RSS hash so tunnelled traffic is
+ distributed across the Rx queues.
+
Removed Items
-------------
diff --git a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
index 26ad105c73..83694133f1 100644
--- a/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
+++ b/drivers/net/dpaa2/base/dpaa2_hw_dpni.c
@@ -381,42 +381,40 @@ dpaa2_distset_to_dpkg_profile_cfg(
case RTE_ETH_RSS_IPV6:
case RTE_ETH_RSS_FRAG_IPV6:
case RTE_ETH_RSS_NONFRAG_IPV6_OTHER:
- case RTE_ETH_RSS_IPV6_EX:
+ case RTE_ETH_RSS_IPV6_EX: {
+ static const uint32_t ip_fields[] = {
+ NH_FLD_IP_SRC, NH_FLD_IP_DST,
+ NH_FLD_IP_PROTO };
+ static const uint8_t ip_hdr_index[] = {
+ 0, (uint8_t)LAST_HDR_INDEX };
+ unsigned int f, h;
if (l3_configured)
break;
l3_configured = 1;
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IP_SRC;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IP_DST;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
-
- kg_cfg->extracts[i].extract.from_hdr.prot =
- NET_PROT_IP;
- kg_cfg->extracts[i].extract.from_hdr.field =
- NH_FLD_IP_PROTO;
- kg_cfg->extracts[i].type =
- DPKG_EXTRACT_FROM_HDR;
- kg_cfg->extracts[i].extract.from_hdr.type =
- DPKG_FULL_FIELD;
- i++;
- break;
+ /* Hash on the outer IP (index 0) and the
innermost
+ * IP instance. A plain frame has a single IP
header,
+ * so only the outer extract resolves; a
tunnelled
+ * frame resolves both and is also spread on
its inner
+ * IP.
+ */
+ for (h = 0; h < RTE_DIM(ip_hdr_index); h++)
+ for (f = 0; f < RTE_DIM(ip_fields);
f++) {
+
kg_cfg->extracts[i].extract.from_hdr.prot =
+ NET_PROT_IP;
+
kg_cfg->extracts[i].extract.from_hdr.hdr_index =
+ ip_hdr_index[h];
+
kg_cfg->extracts[i].extract.from_hdr.field =
+ ip_fields[f];
+ kg_cfg->extracts[i].type =
+ DPKG_EXTRACT_FROM_HDR;
+
kg_cfg->extracts[i].extract.from_hdr.type =
+ DPKG_FULL_FIELD;
+ i++;
+ }
+ break;
+ }
case RTE_ETH_RSS_NONFRAG_IPV4_TCP:
case RTE_ETH_RSS_NONFRAG_IPV6_TCP:
--
2.43.0
