SMMU3.2 brings the support of range-based TLB invalidation and level hint. When this feature is supported, the SMMUv3 driver is allowed to send TLB invalidations for a range of IOVAs instead of using page based invalidation.
Implementing this feature in the virtual SMMUv3 device is mandated for DPDK on guest use case: DPDK uses hugepage buffers and guest sends invalidations for blocks. Without this feature, a guest invalidation of a block of 1GB for instance translates into a storm of page invalidations. Each of them is trapped by the VMM and cascaded downto the physical IOMMU. This completely stalls the execution. This integration issue was initially reported in [1]. Now SMMUv3.2 specifies additional parameters to NH_VA and NH_VAA stage 1 invalidation commands so we can support those extensions. Supporting block mappings in the IOTLB look sensible in terms of TLB entry consumption. However looking at virtio/vhost device usage, without block mapping and without range invalidation (< 5.7 kernels it may be less performant. However for recent guest kernels supporting range invalidations [2], the performance should be similar. Best Regards Eric This series can be found at: https://github.com/eauger/qemu.git branch: v5.1.0-rc1-smmuv3-ril-v4 References: [1] [RFC v2 4/4] iommu/arm-smmu-v3: add CMD_TLBI_NH_VA_AM command for iova range invalidation (https://lists.linuxfoundation.org/pipermail/iommu/2017-August/023679.html [2] 5.7+ kernels featuring 6a481a95d4c1 iommu/arm-smmu-v3: Add SMMUv3.2 range invalidation support History: v3 -> v4: - use tg in hash function and directly compare key fields in the equal function - don't claim 3.2 as we don't have BBML support yet - fixed a bunch of indent issues v2 -> v3: - restore the Jenkins hash function and keep the key as a struct - simplify the logic in smmu_hash_remove_by_asid_iova - added HAD support - expose AIDR (advertise v3.2 support) and fix IIDR offset v1 -> v2: - added "hw/arm/smmu: Introduce smmu_get_iotlb_key()" - removed "[PATCH 5/9] hw/arm/smmuv3: Store the starting level in SMMUTransTableInfo" - Collected Peter's R-b - In this version the key still features TG/LVL. - More details in individual history logs Eric Auger (11): hw/arm/smmu-common: Factorize some code in smmu_ptw_64() hw/arm/smmu-common: Add IOTLB helpers hw/arm/smmu: Introduce smmu_get_iotlb_key() hw/arm/smmu: Introduce SMMUTLBEntry for PTW and IOTLB value hw/arm/smmu-common: Manage IOTLB block entries hw/arm/smmuv3: Introduce smmuv3_s1_range_inval() helper hw/arm/smmuv3: Get prepared for range invalidation hw/arm/smmuv3: Fix IIDR offset hw/arm/smmuv3: Let AIDR advertise SMMUv3.0 support hw/arm/smmuv3: Support HAD and advertise SMMUv3.1 support hw/arm/smmuv3: Advertise SMMUv3.2 range invalidation hw/arm/smmu-internal.h | 8 ++ hw/arm/smmuv3-internal.h | 10 +- include/hw/arm/smmu-common.h | 19 +++- include/hw/arm/smmuv3.h | 1 + hw/arm/smmu-common.c | 214 ++++++++++++++++++++++++----------- hw/arm/smmuv3.c | 142 +++++++++++------------ hw/arm/trace-events | 12 +- 7 files changed, 258 insertions(+), 148 deletions(-) -- 2.21.3
