Am 02.10.2025 um 16:33 hat Thomas Huth geschrieben: > On 02/10/2025 16.04, Kevin Wolf wrote: > > Am 10.09.2025 um 17:37 hat Thomas Huth geschrieben: > > > From: Thomas Huth <[email protected]> > > > > > > When running the tests in thorough mode, e.g. with: > > > > > > make -j$(nproc) check SPEED=thorough > > > > > > we currently always get a huge amount of total tests that the test > > > runner tries to execute (2457 in my case), but a big bunch of them are > > > only skipped (1099 in my case, meaning that only 1358 got executed). > > > This happens because we try to run the whole set of iotests for multiple > > > image formats while a lot of the tests can only run with one certain > > > format only and thus are marked as SKIP during execution. This is quite a > > > waste of time during each test run, and also unnecessarily blows up the > > > displayed list of executed tests in the console output. > > > > > > Thus let's try to be a little bit smarter: If the "check" script is run > > > with "-n" and an image format switch (like "-qed") at the same time (which > > > is what we do already for discovering the tests for the meson test > > > runner), > > > only report the tests that likely support the given format instead of > > > providing the whole list of all tests. We can determine whether a test > > > supports a format or not by looking at the lines in the file that contain > > > a "supported_fmt" or "unsupported_fmt" statement. This is only heuristics, > > > of course, but it is good enough for running the iotests via "make > > > check-block" - I double-checked that the list of executed tests does not > > > get changed by this patch, it's only the tests that are skipped anyway > > > that > > > are now not run anymore. > > > > > > This way the amount of total tests drops from 2457 to 1432 for me, and > > > the amount of skipped tests drops from 1099 to just 74 (meaning that we > > > still properly run 1432 - 74 = 1358 tests as we did before). > > > > > > Signed-off-by: Thomas Huth <[email protected]> > > > --- > > > tests/qemu-iotests/check | 35 ++++++++++++++++++++++++++++++++--- > > > 1 file changed, 32 insertions(+), 3 deletions(-) > > > > > +def dry_run_list(test_dir, imgfmt, testlist): > > > + for t in testlist: > > > + if not imgfmt: > > > + print('\n'.join([os.path.basename(t)])) > > > + continue > > > + # If a format has been given, we look for the "supported_fmt" > > > + # and the "unsupported_fmt" lines in the test and try to find out > > > + # whether the format is supported or not. This is only > > > heuristics, > > > + # but it should be good enough for "make check-block" > > > > I'm not completely sure if this is a good idea at all, but I think we > > should at least mention the possible surprising cases where the > > heuristics fails in this comment. > > Yeah, it's not perfect, but I also failed to come up with a better > solution... do you have any other ideas?
Maybe fixing the qcow vs. qcow2 case could be possible relatively easily by operating only on full words, but I don't have any fundamentally different solution without touching every script. If touching everything is an option, maybe we could have a standardised comment line like we have with "# group:" today. This would be easy to parse and work in both bash and Python scripts. But I'm not sure if it's worth it. > If not, would you be fine with the patch if I respin it with the comment > here extended with some words about those surprising cases? Yes, I just want to have potentially surprising behaviour documented. Kevin
