From: Marek Kasiewicz <[email protected]> Add rte_eth_hdrs_set_mbuf_callback() that allows applications to register a callback providing custom payload mbufs for header split RX mode. When registered, a PMD that supports header split is expected to call this callback at mbuf allocation points to obtain user-provided payload buffers instead of allocating from the mempool.
This enables zero-copy RX for header split: the NIC DMAs the payload directly into application-managed buffers (e.g., mapped frame buffers with known IOVA), bypassing an extra memcpy from the mempool mbuf. A new struct rte_eth_hdrs_mbuf describes the payload buffer (virtual address and IOVA), and the new dev_ops hook hdrs_mbuf_set_cb lets each PMD wire the callback to its receive queue state. The API is marked experimental and exported with version 26.07. Signed-off-by: Marek Kasiewicz <[email protected]> Signed-off-by: Dawid Wesierski <[email protected]> --- lib/ethdev/ethdev_driver.h | 10 +++++++++ lib/ethdev/rte_ethdev.c | 17 ++++++++++++++ lib/ethdev/rte_ethdev.h | 46 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+) diff --git a/lib/ethdev/ethdev_driver.h b/lib/ethdev/ethdev_driver.h index 0f336f9567..b48681268c 100644 --- a/lib/ethdev/ethdev_driver.h +++ b/lib/ethdev/ethdev_driver.h @@ -1292,6 +1292,13 @@ typedef int (*eth_cman_config_set_t)(struct rte_eth_dev *dev, typedef int (*eth_cman_config_get_t)(struct rte_eth_dev *dev, struct rte_eth_cman_config *config); +/** @internal + * Set header split payload mbuf callback for a receive queue. + */ +typedef int (*eth_hdrs_mbuf_set_cb_t)(struct rte_eth_dev *dev, + uint16_t rx_queue_id, void *priv, + rte_eth_hdrs_mbuf_callback_fn cb); + /** * @internal * Dump Rx descriptor info to a file. @@ -1652,6 +1659,9 @@ struct eth_dev_ops { /** Dump Tx descriptor info */ eth_tx_descriptor_dump_t eth_tx_descriptor_dump; + /** Set header split mbuf callback */ + eth_hdrs_mbuf_set_cb_t hdrs_mbuf_set_cb; + /** Get congestion management information */ eth_cman_info_get_t cman_info_get; /** Initialize congestion management structure with default values */ diff --git a/lib/ethdev/rte_ethdev.c b/lib/ethdev/rte_ethdev.c index 9efeaf77cb..d5820ccd22 100644 --- a/lib/ethdev/rte_ethdev.c +++ b/lib/ethdev/rte_ethdev.c @@ -7316,6 +7316,23 @@ rte_eth_ip_reassembly_conf_set(uint16_t port_id, return ret; } +RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_hdrs_set_mbuf_callback, 26.07) +int +rte_eth_hdrs_set_mbuf_callback(uint16_t port_id, uint16_t rx_queue_id, + void *priv, rte_eth_hdrs_mbuf_callback_fn cb) +{ + struct rte_eth_dev *dev; + + RTE_ETH_VALID_PORTID_OR_ERR_RET(port_id, -ENODEV); + dev = &rte_eth_devices[port_id]; + + if (dev->dev_ops->hdrs_mbuf_set_cb == NULL) + return -ENOTSUP; + + return eth_err(port_id, + dev->dev_ops->hdrs_mbuf_set_cb(dev, rx_queue_id, priv, cb)); +} + RTE_EXPORT_EXPERIMENTAL_SYMBOL(rte_eth_dev_priv_dump, 22.03) int rte_eth_dev_priv_dump(uint16_t port_id, FILE *file) diff --git a/lib/ethdev/rte_ethdev.h b/lib/ethdev/rte_ethdev.h index ee400b386f..dbf2c23a35 100644 --- a/lib/ethdev/rte_ethdev.h +++ b/lib/ethdev/rte_ethdev.h @@ -6985,6 +6985,52 @@ rte_eth_tx_buffer(uint16_t port_id, uint16_t queue_id, return rte_eth_tx_buffer_flush(port_id, queue_id, buffer); } +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice. + * + * Buffer descriptor for header split payload mbuf callback. + */ +struct rte_eth_hdrs_mbuf { + void *buf_addr; /**< Virtual address of payload buffer. */ + rte_iova_t buf_iova; /**< IOVA of payload buffer. */ +}; + +/** + * Callback function type for providing custom payload mbufs + * in header split mode. + * + * @param priv + * User-provided private context. + * @param mbuf + * Pointer to buffer descriptor to be filled by the callback. + * @return + * 0 on success, negative errno on failure. + */ +typedef int (*rte_eth_hdrs_mbuf_callback_fn)(void *priv, + struct rte_eth_hdrs_mbuf *mbuf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice. + * + * Register a callback to provide custom payload mbufs for header split RX. + * + * @param port_id + * The port identifier of the Ethernet device. + * @param rx_queue_id + * The index of the receive queue. + * @param priv + * User-provided private context passed to the callback. + * @param cb + * Callback function that provides payload buffer descriptors. + * @return + * 0 on success, negative errno on failure. + */ +__rte_experimental +int rte_eth_hdrs_set_mbuf_callback(uint16_t port_id, uint16_t rx_queue_id, + void *priv, rte_eth_hdrs_mbuf_callback_fn cb); + /** * @warning * @b EXPERIMENTAL: this API may change, or be removed, without prior notice -- 2.47.3 --------------------------------------------------------------------- Intel Technology Poland sp. z o.o. ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN. Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych. Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione. This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.
