With swiotlb=force, the MANA NIC fails to work properly due to commit
730ff06d3f5c ("net: mana: Use page pool fragments for RX buffers instead
of full pages to improve memory efficiency.")Dipayaan tried to fix this by avoiding page pool frags when bounce buffering is in use [1][2]. However, that is not a clean solution: no other NIC drivers need to explicitly check whether bounce buffering is in use. It is also not good for throughput, since dma_map_single()/dma_unmap_single() are then called for each incoming packet. In fact, page pool frags can still be used with the standard MTU of 1500: all we need is to add page_pool_dma_sync_for_cpu() before the CPU reads the incoming packet, so I implemented that in v1 [3]. As Simon suggested [4], this version splits v1 into two patches: Patch 1 adds page_pool_dma_sync_for_cpu(). Patch 2 validates the packet length reported by the NIC. There is no functional difference between v1 and v2, so I am keeping Haiyang's Reviewed-by tag in v2. Please review. Thanks! Note that, with jumbo MTU and XDP, page pool frags are not used, and dma_map_single()/dma_unmap_single() are still called for each incoming packet, causing poor throughput with swiotlb=force; see mana_get_rxbuf_cfg() and mana_refill_rx_oob() -> mana_get_rxfrag(). The jumbo MTU/XDP issue will be addressed later since that needs more consideration if we want to use page pool with PP_FLAG_DMA_MAP there: e.g., for XDP, the received packet can be transmitted in place, i.e. the same RX buffer can be used as a TX buffer: mana_rx_skb() -> mana_xdp_tx() -> mana_start_xmit() -> mana_map_skb(). In mana_create_page_pool(), we may have to set pprm.dma_dir to DMA_BIDIRECTIONAL if XDP is in use: pprm.dma_dir = mana_xdp_get(mpc) ? DMA_BIDIRECTIONAL : DMA_FROM_DEVICE; In the case of XDP, the next issue is that mana_rx_skb() -> ... -> mana_map_skb() appears to call dma_map_single() on an RX buffer allocated from a page pool created with PP_FLAG_DMA_MAP, which seems incorrect. Any thoughts? [1] https://lore.kernel.org/all/ae91hyrlf4n23...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/#r [2] https://lore.kernel.org/all/ae9pxvjfkazyf...@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net/ [3] https://lore.kernel.org/all/[email protected]/ [4] https://lore.kernel.org/all/[email protected]/ Dexuan Cui (2): net: mana: Sync page pool RX frags for CPU net: mana: Validate the packet length reported by the NIC drivers/net/ethernet/microsoft/mana/mana_en.c | 61 +++++++++++++++---- include/net/mana/mana.h | 8 +++ 2 files changed, 58 insertions(+), 11 deletions(-) -- 2.34.1
