From: Kunwu Chan <[email protected]>
ARM SPE and other tracing PMUs write records directly to perf AUX
buffers. Unlike ordinary sampling PMUs, they do not necessarily
generate a perf_event_overflow() callback for each record. A kernel
consumer therefore needs to own and drain an AUX buffer rather than
rely on the overflow path.
Userspace obtains AUX storage through perf_event_open() and mmap().
Kernel events created by perf_event_create_kernel_counter() have no
public interface for allocating that storage: rb_alloc_aux() is
reached through perf_mmap().
This series adds a small perf-core API for that use case. Patches 1-2
add an in-kernel AUX owner, a setup/release pair, and producer-head,
consumer-tail, and copy helpers without exposing perf's internal page
array. Patches 3-5 add KUnit tests, a userspace regression test, and
a selftest orchestration script. The first expected consumer is a
DAMON ARM SPE backend, but this series is independent of DAMON and
does not add consumer-specific policy to perf.
The API follows the existing non-overwrite AUX protocol:
- setup validates the event, rejects non-power-of-two page counts and
negative watermark, allocates the ring buffer and AUX pages, and
attaches through perf's mmap-serialized lifecycle;
- release stops writers, drops the event->rb reference, and detaches
only a buffer owned by this API;
- head reads use smp_rmb() to pair with the producer's data-write
barrier;
- tail updates use smp_rmb() before and smp_mb() after to order data
reads before releasing space;
- copy validates the requested modular cursor interval against
[tail, head] before applying the ring mask and holds an AUX
reference while copying.
Testing
=======
Based on commit 917d558b151c ("perf/x86: Optimize ACR handling in
match_prev_assignment()") in the tip perf/core branch.
Built and tested on arm64 (Kunpeng 920, ARM SPE present). Each commit
compiles independently. Full Image build succeeds with
CONFIG_PERF_AUX_KERNEL_KUNIT_TEST=y.
KUnit (39 cases, dummy AUX PMU, zero hardware dependency):
pass:39 fail:0 skip:0
The dummy PMU registered in suite_init provides ->setup_aux/->free_aux
callbacks; setup_aux pre-fills pages with 0xAB so copy tests verify
data correctness byte-by-byte. Cases cover:
- setup/release lifecycle (12): non-power-of-2 rejection, pow2
acceptance, double-setup -EBUSY, parent event rejection, negative
watermark, explicit watermark, writer admission/blocking, no-op
release, setup/release roundtrip, multi-cycle, double release
- accessor contract (20): head initial/advances/no-rb, copy full
window/wrap/zero-len/data-correctness, copy rejects
rewound/future/oversized/null/already-consumed/no-rb, tail_set
valid/future/within-window/behind-ring/noop/consume-all/no-rb
- multi-cycle produce->copy->tail_set (1)
- perf_aux_output_end size=0, buffer-full stops producer (2)
- two-event independence, release isolation (2)
- user/kernel coexistence (1): simulated aux_mmap_count=1 event
and aux_kernel_count=1 event on the same PMU, verify mutual
non-interference and release isolation
- concurrent release (1): two kthreads concurrently call
perf_event_release_aux() on the same event, verifying the
refcount_dec_and_mutex_lock serialisation and detach-block
event->rb guard correctly handle the race
Selftest script (9 sections):
94 passed, 0 failed, 2 skipped -- Overall: PASS
Sections: build-time config, KUnit orchestration, userspace
regression, hardware perf record + dmesg, DAMON integration,
memory ordering static analysis, boundary validation, concurrency
safety, user/kernel AUX isolation.
Userspace C program:
9 PASS, 0 FAIL, 2 SKIP
The program follows the standard dual-mmap AUX protocol: mmap the
metadata/data ring at offset 0, set aux_offset/aux_size in the
metadata page, then mmap the AUX area at aux_offset. When the AUX
area mmap fails (e.g. PMU does not support it), the test is SKIP
rather than FAIL. ARM SPE single-event limitation is also SKIP.
Kunwu Chan (3):
perf/core: add AUX buffer ownership for kernel events
perf/core: add AUX ring accessors for kernel consumers
selftests/perf_events: add AUX kernel API selftest script
Lian Wang (2):
perf/core: add KUnit tests for AUX kernel-consumer API
selftests/perf_events: add userspace AUX regression test
.../userspace-api/perf_ring_buffer.rst | 56 +-
include/linux/perf_event.h | 18 +
kernel/events/Makefile | 1 +
kernel/events/aux_kernel_test.c | 1144 +++++++++++++++++
kernel/events/core.c | 139 ++
kernel/events/internal.h | 1 +
kernel/events/ring_buffer.c | 205 ++-
lib/Kconfig.debug | 15 +
tools/testing/selftests/perf_events/Makefile | 2 +
.../selftests/perf_events/aux_kernel.sh | 563 ++++++++
.../perf_events/aux_kernel_usermode.c | 546 ++++++++
tools/testing/selftests/perf_events/config | 2 +
12 files changed, 2687 insertions(+), 5 deletions(-)
create mode 100644 kernel/events/aux_kernel_test.c
create mode 100755 tools/testing/selftests/perf_events/aux_kernel.sh
create mode 100644 tools/testing/selftests/perf_events/aux_kernel_usermode.c
--
2.43.0
