https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111044
Bug ID: 111044
Summary: [OpenMP] Use additional kinds of libmemkind for
allocators; make libgomp.texi doc more explicit
Product: gcc
Version: 14.0
Status: UNCONFIRMED
Keywords: openmp
Severity: normal
Priority: P3
Component: libgomp
Assignee: unassigned at gcc dot gnu.org
Reporter: burnus at gcc dot gnu.org
CC: jakub at gcc dot gnu.org
Target Milestone: ---
Motivated by some investigations related to
PR111024 - libgomp: FAILs with oldish libnuma/libmemkind
* * *
Expected:
- Use additional memkind kinds when it make sense and they are available
- Make libgomp.texi description more explicit
- audit whether the current choices make sense
See below for details.
* * *
Looking at memkind library, I see that we have the following; I wonder whether
we additional memkinds should be used/add in libgomp.
(X - used in libgomp/allocator.c)
(S/E - standard / experimental API according to comment in memkind.h)
(libnuma version)
See also https://github.com/memkind/memkind
X S 1.10 MEMKIND_DAX_KMEM (omp_large_cap_mem_space if _ALL not existing)
X S 1.10 MEMKIND_DAX_KMEM_ALL (omp_large_cap_mem_space)
- S 1.10 MEMKIND_DAX_KMEM_PREFERRED
- S 1.11 MEMKIND_DAX_KMEM_INTERLEAVE
- S 1.11 MEMKIND_HIGHEST_CAPACITY
- S 1.11 MEMKIND_HIGHEST_CAPACITY_PREFERRED
- S 1.11 MEMKIND_HIGHEST_CAPACITY_LOCAL
- S 1.11 MEMKIND_HIGHEST_CAPACITY_LOCAL_PREFERRED
- S 1.11 MEMKIND_LOWEST_LATENCY_LOCAL
- S 1.11 MEMKIND_LOWEST_LATENCY_LOCAL_PREFERRED
- S 1.11 MEMKIND_HIGHEST_BANDWIDTH_LOCAL
- S 1.11 MEMKIND_HIGHEST_BANDWIDTH_LOCAL_PREFERRED
- E 1.6 MEMKIND_REGULAR
X E 0.1 MEMKIND_DEFAULT
- E 1.6 MEMKIND_HUGETLB
- E 0.1 MEMKIND_HBW
- E 1.6 MEMKIND_HBW_ALL
X E 0.1 MEMKIND_HBW_PREFERRED (omp_high_bw_mem_space)
- E 0.1 MEMKIND_HBW_HUGETLB
- E 1.6 MEMKIND_HBW_ALL_HUGETLB
- E 0.1 MEMKIND_HBW_PREFERRED_HUGETLB
X E 0.3 MEMKIND_HBW_INTERLEAVE (omp_high_bw_mem_space + omp_atv_interleaved)
X E 0.3 MEMKIND_INTERLEAVE (omp_atv_interleaved)
It looks as if we could at least add MEMKIND_DAX_KMEM_INTERLEAVE for
omp_large_cap_mem_space + omp_atv_interleaved-
I also wonder whether we should try MEMKIND_HIGHEST_CAPACITY(_ALL) - when
persistent memory MEMKIND_DAX_KMEM(_ALL) is not available.
For omp_atv_nearest, I see high cap + low latency _LOCAL_PREFERRED variants
(i.e. use local if available, otherwise fall back to other memory) or _LOCAL
(use local if available, return NULL if not enough space.)
And MEMKIND_HIGHEST_BANDWIDTH_LOCAL(_PREFERRED) would be a omp_atv_nearest
alternative to MEMKIND_HBW.
Currently, omp_atv_nearest (unless any memkind triggers) uses libnuma's
numa_alloc_local.
PLUS:
https://gcc.gnu.org/onlinedocs/libgomp/Memory-allocation.html should be more
explicit what is used with libnuma and especially libmemkind - it currently
lists the algorithm but not the used MEMKIND_* kinds.
