Hi Roger, > On Nov 7, 2023, at 00:06, Andrew Cooper <[email protected]> wrote: > > On 06/11/2023 2:27 pm, Roger Pau Monne wrote: >> The current implementation of x2APIC requires to either use Cluster Logical >> or >> Physical mode for all interrupts. However the selection of Physical vs >> Logical >> is not done at APIC setup, an APIC can be addressed both in Physical or >> Logical >> destination modes concurrently. >> >> Introduce a new x2APIC mode called Mixed, which uses Logical Cluster mode for >> IPIs, and Physical mode for external interrupts, thus attempting to use the >> best method for each interrupt type. >> >> Using Physical mode for external interrupts allows more vectors to be used, >> and >> interrupt balancing to be more accurate. >> >> Using Logical Cluster mode for IPIs allows fewer accesses to the ICR register >> when sending those, as multiple CPUs can be targeted with a single ICR >> register >> write. >> >> A simple test calling flush_tlb_all() 10000 times on a tight loop on AMD EPYC >> 9754 with 512 CPUs gives the following figures in nano seconds: >> >> x mixed >> + phys >> * cluster >> N Min Max Median Avg Stddev >> x 25 3.5131328e+08 3.5716441e+08 3.5410987e+08 3.5432659e+08 1566737.4 >> + 12 1.231082e+09 1.238824e+09 1.2370528e+09 1.2357981e+09 2853892.9 >> Difference at 95.0% confidence >> 8.81472e+08 +/- 1.46849e+06 >> 248.774% +/- 0.96566% >> (Student's t, pooled s = 2.05985e+06) >> * 11 3.5099276e+08 3.5561459e+08 3.5461234e+08 3.5415668e+08 1415071.9 >> No difference proven at 95.0% confidence >> >> So Mixed has no difference when compared to Cluster mode, and Physical mode >> is >> 248% slower when compared to either Mixed or Cluster modes with a 95% >> confidence. >> >> Note that Xen uses Cluster mode by default, and hence is already using the >> fastest way for IPI delivery at the cost of reducing the amount of vectors >> available system-wide. >> >> Make the newly introduced mode the default one. >> >> Note the printing of the APIC addressing mode done in connect_bsp_APIC() has >> been removed, as with the newly introduced mixed mode this would require more >> fine grained printing, or else would be incorrect. The addressing mode can >> already be derived from the APIC driver in use, which is printed by different >> helpers. >> >> Suggested-by: Andrew Cooper <[email protected]> >> Signed-off-by: Roger Pau Monné <[email protected]> > > Reviewed-by: Andrew Cooper <[email protected]>
Acked-by: Henry Wang <[email protected]> # CHANGELOG Kind regards, Henry
