Hi Slava, Thanks for the review.
On 2026/8/6 15:56, Slava Ovsiienko wrote:
Hi, Why does cross-NUMA pinning cause the huge page size alignment?
In practice, scenarios involving intra-NUMA binding encounter issues with huge-page alignment.
Even with the loop bug (alignment block always entered), there is a second issue: the if (reg_end < hugepage_end) gate uses strict < instead of <=. Runtime data with 1 GB hugepages on Kunpeng SoC + CX7 (NUMA1), MPRQ enabled, 16 cores, 16 queues, queue-size 4096: intra-NUMA (-l 40-55, all on NUMA1): reg_start = 0x121b76f000 reg_end = 0x1240000000 hugepage_start = 0x1200000000 hugepage_end = 0x1240000000 Here reg_end == hugepage_end, so reg_end < hugepage_end is false. Alignment is skipped, reg_start stays at 0x121b76f000 (not aligned to the 1 GB boundary) → high TLB miss rate under SMMU. cross-NUMA (-l 11-26, all on NUMA0, CX7 on NUMA1): reg_start = 0x11d93b5000 reg_end = 0x11fdc46000 hugepage_start = 0x11c0000000 hugepage_end = 0x1200000000 Here reg_end (0x11fdc46000) < hugepage_end (0x1200000000) is true. Alignment runs: reg_start → hugepage_start, reg_end → hugepage_end → MR is hugepage-aligned → no excessive TLB misses. In both cases the mempool (~585 MB) fits within a single 1 GB hugepage and reg_start is NOT aligned. The only difference is whether reg_end happens to coincide with hugepage_end. The intra-NUMA case happens to fill the hugepage to its exact end, which makes the < check fail. The mempool is always allocated on NUMA1 via rte_mempool_create with numa_node = dev->device->numa_node. We don't yet have a definitive explanation for why the placement differs between the two pinning configurations — it likely depends on the hugepage allocator's internal state. But the key point is that alignment should not depend on this coincidence; reg_start should be aligned regardless of where reg_end falls.
It seems we have a bug in the current code (and the patch fixes it somehow ) - the loop "/* Check that all ranges are on pages of the same size. */" does not depend on iteration variable "I" and always runs till the end.
You are right about the loop bug — the original loop always runs to completion because it uses msl->page_sz (fetched once before the loop) instead of each range's own memseg list. Our patch fixes this by calling rte_mem_virt2memseg_list() inside the loop.
The patch mostly looks good to me, there are just my 5 cents to improve: 1.msl = rte_mem_virt2memseg_list((void *)ranges[0].start);We could move this into the checking loop inside (prefertable). Or start the loop from i=1. Just to have rte_mem_virt2memseg_list() call in single point.if (msl != NULL) { - uint64_t hugepage_sz = 0; + hugepage_sz = msl->page_sz;
2.+ if (same_hugepage_sz && hugepage_sz > 0) { + unsigned int orig_ranges_n = ranges_n;No sense to run this code if hugepage_sz is equal to rte_mem_page_size (), all alignment and range merging is done by mlx5_mempool_get_chunks(). 3. Also, I would be careful about external memory. All ranges are already aligned to the correct page sizes, so let's add "if !is_extmem" checking.
I will update the version based on your suggestions. Thanks, Xingui
