On Thu, Jul 24, 2025 at 2:00 PM Alex Bennée <alex.ben...@linaro.org> wrote: > > As our set of multiarch tests has grown the practice of running every > plugin with every test is becoming unsustainable. If we switch to > ensuring every test gets run with at least one plugin we can speed > things up. > > Some plugins do need to be run with specific tests (for example the > memory instrumentation test). We can handle this by manually adding > them to EXTRA_RUNS. We also need to wrap rules in a CONFIG_PLUGIN test > so we don't enable the runs when plugins are not enabled. > > Signed-off-by: Alex Bennée <alex.ben...@linaro.org> > --- > tests/tcg/Makefile.target | 23 ++++++++++++++----- > tests/tcg/multiarch/Makefile.target | 8 +++++-- > .../multiarch/system/Makefile.softmmu-target | 11 +++++---- > 3 files changed, 30 insertions(+), 12 deletions(-) > > diff --git a/tests/tcg/Makefile.target b/tests/tcg/Makefile.target > index a12b15637ea..18afd5be194 100644 > --- a/tests/tcg/Makefile.target > +++ b/tests/tcg/Makefile.target > @@ -173,14 +173,25 @@ PLUGINS=$(filter-out $(DISABLE_PLUGINS), \ > # We need to ensure expand the run-plugin-TEST-with-PLUGIN > # pre-requistes manually here as we can't use stems to handle it. We > # only expand MULTIARCH_TESTS which are common on most of our targets > -# to avoid an exponential explosion as new tests are added. We also > -# add some special helpers the run-plugin- rules can use below. > +# and rotate the plugins so we don't grow too out of control as new > +# tests are added. Plugins that need to run with a specific test > +# should ensure they add their combination to EXTRA_RUNS. > > ifneq ($(MULTIARCH_TESTS),) > -$(foreach p,$(PLUGINS), \ > - $(foreach t,$(MULTIARCH_TESTS),\ > - $(eval run-plugin-$(t)-with-$(p): $t $p) \ > - $(eval RUN_TESTS+=run-plugin-$(t)-with-$(p)))) > + > +NUM_PLUGINS := $(words $(PLUGINS)) > +NUM_TESTS := $(words $(MULTIARCH_TESTS)) > + > +define mod_plus_one > + $(shell $(PYTHON) -c "print( ($(1) % $(2)) + 1 )") > +endef > + > +$(foreach _idx, $(shell seq 1 $(NUM_TESTS)), \ > + $(eval _test := $(word $(_idx), $(MULTIARCH_TESTS))) \ > + $(eval _plugin := $(word $(call mod_plus_one, $(_idx), > $(NUM_PLUGINS)), $(PLUGINS))) \ > + $(eval run-plugin-$(_test)-with-$(_plugin): $(_test) $(_plugin)) \ > + $(eval RUN_TESTS+=run-plugin-$(_test)-with-$(_plugin))) > + > endif # MULTIARCH_TESTS > endif # CONFIG_PLUGIN > > diff --git a/tests/tcg/multiarch/Makefile.target > b/tests/tcg/multiarch/Makefile.target > index bfdf7197a7b..38345ff8805 100644 > --- a/tests/tcg/multiarch/Makefile.target > +++ b/tests/tcg/multiarch/Makefile.target > @@ -189,6 +189,10 @@ run-plugin-semiconsole-with-%: > TESTS += semihosting semiconsole > endif > > +test-plugin-mem-access: CFLAGS+=-pthread -O0 > +test-plugin-mem-access: LDFLAGS+=-pthread -O0 > + > +ifeq ($(CONFIG_PLUGIN),y) > # Test plugin memory access instrumentation > run-plugin-test-plugin-mem-access-with-libmem.so: \ > PLUGIN_ARGS=$(COMMA)print-accesses=true > @@ -197,8 +201,8 @@ run-plugin-test-plugin-mem-access-with-libmem.so: \ > $(SRC_PATH)/tests/tcg/multiarch/check-plugin-output.sh \ > $(QEMU) $< > > -test-plugin-mem-access: CFLAGS+=-pthread -O0 > -test-plugin-mem-access: LDFLAGS+=-pthread -O0 > +EXTRA_RUNS += run-plugin-test-plugin-mem-access-with-libmem.so > +endif > > # Update TESTS > TESTS += $(MULTIARCH_TESTS) > diff --git a/tests/tcg/multiarch/system/Makefile.softmmu-target > b/tests/tcg/multiarch/system/Makefile.softmmu-target > index 5acf2700812..4171b4e6aa0 100644 > --- a/tests/tcg/multiarch/system/Makefile.softmmu-target > +++ b/tests/tcg/multiarch/system/Makefile.softmmu-target > @@ -71,8 +71,11 @@ endif > MULTIARCH_RUNS += run-gdbstub-memory run-gdbstub-interrupt \ > run-gdbstub-untimely-packet run-gdbstub-registers > > +ifeq ($(CONFIG_PLUGIN),y) > # Test plugin memory access instrumentation > -run-plugin-memory-with-libmem.so: \ > - PLUGIN_ARGS=$(COMMA)region-summary=true > -run-plugin-memory-with-libmem.so: \ > - > CHECK_PLUGIN_OUTPUT_COMMAND=$(MULTIARCH_SYSTEM_SRC)/validate-memory-counts.py > $@.out > +run-plugin-memory-with-libmem.so: memory libmem.so
Hm why wasn't this needed before this change? I see the make `memory` target sets CHECK_UNALIGNED but where is libmem.so target coming from? > +run-plugin-memory-with-libmem.so: PLUGIN_ARGS=$(COMMA)region-summary=true > +run-plugin-memory-with-libmem.so: > CHECK_PLUGIN_OUTPUT_COMMAND=$(MULTIARCH_SYSTEM_SRC)/validate-memory-counts.py > $@.out > + > +EXTRA_RUNS += run-plugin-memory-with-libmem.so > +endif > -- > 2.47.2 > >
