On Thu, Sep 24, 2026 at 5:56 AM David Woodhouse <[email protected]> wrote: > > On Thu, 2026-09-24 at 00:26 +0800, KunWu Chan wrote: > > The parallel interrupt delivery case sounds particularly interesting. > > Please don't go to any trouble if the old results are not available. > > The old results were on the original RCU-based (not SRCU) version, so > I redid them with the latest, on the 192-way PREEMPT_RT GNR box. > > It's a bit of a microbenchmark: N vCPUs spinning in guest mode, with N > more sender threads, each delivering interrupts as fast as it can to > its own paired vCPU thread. The vCPU isn't even acknowledging the > interrupt, but the locked btsl will still be dirtying the cache lines > of the shared info evtchn_pending bitmap even to find that the bit is > already set (qv). > > The rwlock version gets up to 8-9M deliveries per second and then > starts to regress past about 32 parallel vCPUs. While SRCU starts off > faster, and gets more so. > > Deliveries/sec, spread layout: > ┌───────┬──────────┬─────────────┬───────┐ > │ vCPUs │ rwlock │ atomic SRCU │ ratio │ > ├───────┼──────────┼─────────────┼───────┤ > │ 1 │ 1.02M │ 1.13M │ 1.1× │ > ├───────┼──────────┼─────────────┼───────┤ > │ 2 │ 1.87M │ 2.27M │ 1.2× │ > ├───────┼──────────┼─────────────┼───────┤ > │ 4 │ 3.59M │ 4.57M │ 1.3× │ > ├───────┼──────────┼─────────────┼───────┤ > │ 8 │ 7.06M │ 9.13M │ 1.3× │ > ├───────┼──────────┼─────────────┼───────┤ > │ 16 │ 8.75M │ 16.8M │ 1.9× │ > ├───────┼──────────┼─────────────┼───────┤ > │ 32 │ 8.52M │ 33.3M │ 3.9× │ > ├───────┼──────────┼─────────────┼───────┤ > │ 64 │ 6.75M │ 63.7M │ 9.4× │ > └───────┴──────────┴─────────────┴───────┘ > > This is with the port numbers of each of the N CPUs deliberately spread > out into different cache lines (which arguably we ought to do for > production guests, but we don't). If I let them all pack with > sequential port# allocation, it makes no difference to rwlock but the > SRCU variant hits a scaling cliff around N=16 too (17.2M @32, 14.0M > @64). But hey, as I said: microbenchmark.
Resend. Hi David, Thanks for taking the time to rerun this. This is exactly the kind of data I was hoping to get, and the scaling at 32 and 64 vCPUs is particularly useful. The cache-line layout comparison is also a useful data point. I'll keep the microbenchmark limitation in mind when using these results. Thanks again for the extra work. Thanks, Kunwu
