On Tue, 22 Sep 2026 12:37:43 +0200 David Woodhouse <[email protected]> wrote:
> On Tue, 2026-09-22 at 11:16 +0800, KunWu Chan wrote: > > Do you happen to have any numbers comparing the GPC invalidation > > latency with regular SRCU vs. `synchronize_srcu_atomic()`? If there > > are also numbers with the reader-free fastpath, that would be useful > > for understanding its impact as well. > > Yeah, I built some latency tests and was posting results in the earlier > thread¹, on a few different test hosts. > > I compared against the existing rwlock, as well as SRCU both with and > without the try_synchronize_srcu() fast path. Mostly looking at the > invalidation latency, since that was Sean's stated concern with the > original RCU-based proof of concept. > > All from the same test: 12 concurrent guest-memory invalidation > reproducers hammering the Xen shinfo/vcpu_info caches, 300 second > windows, measuring the invalidation drain end-to-end. > > 192-way Granite Rapids, PREEMPT_RT production config: > > rwlock (before this series) avg 4.4µs max 3.85ms > synchronize_srcu_expedited() drain avg 8.6µs max 810µs > synchronize_srcu_atomic() + fastpath avg ~3µs max 801µs > > The A/B numbers I have for the reader-free fast path were on different > hardware (128-way Ice Lake, production-like config): > > synchronize_srcu_atomic(), no fastpath avg 8.0µs max 6.0ms > with the inline no-readers proof avg 3.6µs max 326µs > > If you want, it isn't much effort for me to tell my friend to redo any > of the measurements. > > ¹ > https://lore.kernel.org/all/[email protected]/ > Hi David, Thanks, this is very helpful. I've put the results together below. KVM GPC invalidation drain latency 128-way Ice Lake: ┌──────────────────────────────────┬──────────────┬──────────────┐ │ Implementation │ Avg │ Max │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ Baseline (rwlock) │ not provided │ not provided │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ synchronize_srcu_expedited() │ not provided │ not provided │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ synchronize_srcu_atomic() │ 8.0us │ 6.0ms │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ synchronize_srcu_atomic() + │ 3.6us │ 326us │ │ reader-free fastpath │ │ │ └──────────────────────────────────┴──────────────┴──────────────┘ 192-way Granite Rapids: ┌──────────────────────────────────┬──────────────┬──────────────┐ │ Implementation │ Avg │ Max │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ Baseline (rwlock) │ 4.4us │ 3.85ms │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ synchronize_srcu_expedited() │ 8.6us │ 810us │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ synchronize_srcu_atomic() │ not provided │ not provided │ ├──────────────────────────────────┼──────────────┼──────────────┤ │ synchronize_srcu_atomic() + │ ~3us │ 801us │ │ reader-free fastpath │ │ │ └──────────────────────────────────┴──────────────┴──────────────┘ Note: Avg / Max are the average and maximum end-to-end invalidation drain latency. The measurements use 12 concurrent guest-memory invalidation reproducers hammering the Xen shinfo/vcpu_info caches over 300-second windows. On the 128-way Ice Lake system, the reader-free fastpath reduces the average latency from 8.0us to 3.6us, and the maximum from 6.0ms to 326us. The 192-way Granite Rapids result is from a separate hardware configuration, so I kept it separate from the 128-way A/B comparison. The only missing comparison is the 192-way Granite Rapids result for synchronize_srcu_atomic() without the reader-free fastpath. If you already have that result, it would be useful to add it. No need to rerun the measurement just for this table if you don't have it. Could you please confirm that I transcribed the numbers correctly? Thanks, Kunwu
