Runtime queue setup on E810 VFs causes queue state corruption when
queues are dynamically reconfigured while the hardware rate limiter
is actively pacing TX queues. Queue configuration messages to the PF
via virtchnl can race with ongoing TX operations, leading to undefined
behavior.
Rather than gating this behind a devarg that an application would have
to know to set (and could just as easily avoid triggering the race by
not calling rte_eth_{rx,tx}_queue_setup() on a running port), stop
advertising RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP and
RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP as soon as the application
commits a per-queue bandwidth rte_tm hierarchy, i.e. as soon as the
condition that causes the race actually exists. iavf_dev_info_get() is
re-queried by the ethdev layer on every rx/tx_queue_setup() call, so
this is enough for the generic layer to start rejecting runtime queue
(re)configuration with -EBUSY once queue rate limiting is active, and
to automatically allow it again once the rte_tm hierarchy is torn
down.
vf->qtc_map, already used elsewhere to look up a queue's TC mapping,
is repurposed as the "queue bandwidth committed" signal since it's set
by iavf_hierarchy_commit() exactly when a per-queue bandwidth mapping
has been pushed to the PF, regardless of whether the port was stopped
at the time. Fix two related issues found while making it load-bearing
for this check:
- iavf_hierarchy_commit() replaced vf->qtc_map on every successful
commit without freeing the previous allocation, leaking memory.
- vf->qtc_map was never released on VF teardown, so
iavf_uninit_vf()/iavf_init_vf() (e.g. across a VF reset) could leave
a stale pointer referencing freed unrelated memory, and the runtime
queue setup capability would never be re-advertised after a reset.
Both are fixed by freeing vf->qtc_map before replacing it in
iavf_hierarchy_commit(), and freeing and clearing it in
iavf_uninit_vf().
Signed-off-by: Marek Kasiewicz <[email protected]>
Signed-off-by: Dawid Wesierski <[email protected]>
---
doc/guides/nics/intel_vf.rst | 10 ++++++++++
doc/guides/rel_notes/release_26_07.rst | 3 +++
drivers/net/intel/iavf/iavf_ethdev.c | 23 ++++++++++++++++++++---
drivers/net/intel/iavf/iavf_tm.c | 2 ++
4 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/doc/guides/nics/intel_vf.rst b/doc/guides/nics/intel_vf.rst
index e010f852cf..a47e3f6736 100644
--- a/doc/guides/nics/intel_vf.rst
+++ b/doc/guides/nics/intel_vf.rst
@@ -150,6 +150,16 @@ Intel\ |reg| E800 Series Ethernet devices:
for example: ``-a 18:00.0,quanta_size=2048``.
The default value is 1024, and quanta size should be set as the product of
64 in legacy host interface mode.
+* Runtime (post-start) Rx/Tx queue setup can race with the hardware Tx rate
+ limiter on E810 VFs and corrupt queue state.
+ Once an application commits a per-queue bandwidth ``rte_tm`` hierarchy,
+ the driver automatically stops advertising
+ ``RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP`` and
+ ``RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP``,
+ so ``rte_eth_rx_queue_setup()``/``rte_eth_tx_queue_setup()``
+ are rejected with ``-EBUSY`` on a running port for as long as queue rate
+ limiting is active.
+
* When using the Intel out-of-tree "ice" PF/kernel driver v1.13.7 or later,
to create VFs with >16 queues (aka. "large VFs"),
it is necessary to change the rss_lut_vf_addr setting in sysfs from the
default of 64 to 512.
diff --git a/doc/guides/rel_notes/release_26_07.rst
b/doc/guides/rel_notes/release_26_07.rst
index 6badd6d91b..4faa7b8fe2 100644
--- a/doc/guides/rel_notes/release_26_07.rst
+++ b/doc/guides/rel_notes/release_26_07.rst
@@ -144,6 +144,9 @@ New Features
* Added support for QinQ offloading operations.
* Added support for transmitting LLDP packets based on mbuf packet type.
* Implemented AVX2 context descriptor transmit paths.
+ * Runtime Rx/Tx queue setup is now automatically disabled while a
+ per-queue bandwidth ``rte_tm`` hierarchy is committed, to avoid
+ corrupting queue state on E810 VFs.
* **Updated Intel ice driver.**
diff --git a/drivers/net/intel/iavf/iavf_ethdev.c
b/drivers/net/intel/iavf/iavf_ethdev.c
index d601ec3b6a..e475b64971 100644
--- a/drivers/net/intel/iavf/iavf_ethdev.c
+++ b/drivers/net/intel/iavf/iavf_ethdev.c
@@ -1160,9 +1160,18 @@ iavf_dev_info_get(struct rte_eth_dev *dev, struct
rte_eth_dev_info *dev_info)
dev_info->reta_size = vf->vf_res->rss_lut_size;
dev_info->flow_type_rss_offloads = IAVF_RSS_OFFLOAD_ALL;
dev_info->max_mac_addrs = IAVF_NUM_MACADDR_MAX;
- dev_info->dev_capa =
- RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
- RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
+ /*
+ * Runtime queue setup can race with the hardware Tx rate limiter on
+ * E810 VFs and corrupt queue state. Once a per-queue bandwidth rte_tm
+ * hierarchy has been committed (vf->qtc_map is set), stop advertising
+ * the capability so the ethdev layer rejects further
rx/tx_queue_setup()
+ * calls on a running port with -EBUSY. The capability is re-advertised
+ * automatically once the rte_tm hierarchy is torn down.
+ */
+ if (vf->qtc_map == NULL)
+ dev_info->dev_capa =
+ RTE_ETH_DEV_CAPA_RUNTIME_RX_QUEUE_SETUP |
+ RTE_ETH_DEV_CAPA_RUNTIME_TX_QUEUE_SETUP;
dev_info->rx_offload_capa =
RTE_ETH_RX_OFFLOAD_VLAN_STRIP |
RTE_ETH_RX_OFFLOAD_QINQ_STRIP |
@@ -2760,6 +2769,14 @@ iavf_uninit_vf(struct rte_eth_dev *dev)
rte_free(vf->proto_xtr);
vf->proto_xtr = NULL;
+ /*
+ * Drop the committed queue/TC bandwidth mapping so a subsequent
+ * iavf_init_vf() (e.g. after a device reset) starts with runtime
+ * Rx/Tx queue setup available again (see iavf_dev_info_get()).
+ */
+ rte_free(vf->qtc_map);
+ vf->qtc_map = NULL;
+
rte_free(vf->rss_lut);
vf->rss_lut = NULL;
rte_free(vf->rss_key);
diff --git a/drivers/net/intel/iavf/iavf_tm.c b/drivers/net/intel/iavf/iavf_tm.c
index faa2d4b8a0..fc84bf7e6b 100644
--- a/drivers/net/intel/iavf/iavf_tm.c
+++ b/drivers/net/intel/iavf/iavf_tm.c
@@ -912,6 +912,8 @@ static int iavf_hierarchy_commit(struct rte_eth_dev *dev,
if (ret_val)
goto fail_clear;
+ /* replace the previously committed mapping, if any */
+ rte_free(vf->qtc_map);
vf->qtc_map = qtc_map;
free(old_qtc_map);
if (adapter->stopped == 1)
--
2.47.3
---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial
Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 |
Kapital zakladowy 200.000 PLN.
Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z
dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach
handlowych.
Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i
moze zawierac informacje poufne. W razie przypadkowego otrzymania tej
wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie;
jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). If you are not the intended recipient, please
contact the sender and delete all copies; any review or distribution by others
is strictly prohibited.
