As promised, here are the changes to move most of the migration tests
into a separate check-migration target that is intended to be used by
migration maintainers and contributors.
A few tests will still be kept in another check-migration-quick
target, that will run as part of make check by default. Those are
there to catch changes in generic code that might affect migration and
the rare device code change that affects migration ("rare" because
migration-test runs with only default devices).
The are two main reasons for this:
- Reduce the time of the common 'make check' invocation.
migration-test is notoriously wasteful in that it runs a migration
for every test. We don't yet have a comprehensive coverage analysis
nor the mechanisms for testing isolated parts of the code without
resorting to a full migration.
While the migration tests don't account for that much of make check
time in a parallel run (only about 30s, see table below), I'd like
to be able to write tests without encumbering every single make
check run.
- Avoid breaking a developer's make check invocation and also CI when
only non-migration changes are present.
The migration tests are subject to races in the code that tend to be
quite intermittent. Ideally, if a code change does not touch
migration, it should not break migration-test.
make -j16 before after migration suite included
--------------------------------------------------------------------------
check 2m28.923s 1m56.178s migration-quick
check SPEED=slow 6m07.854s 5m59.982s migration
check SPEED=thorough 10m17.102s 10m02.416s migration
check-qtest 2m07.827s 1m24.065s none
check-qtest SPEED=slow 5m06.149s 4m17.897s none
check-qtest SPEED=thorough 5m7.601s 4m18.135s none
check-migration 3m41.927s migration
check-migration-quick 0m15.592s migration-quick
--------------------------------------------------------------------------
Note that this series could very well be just patch 3 without any new
target, but then to be able to run the full set of migration tests
we'd need to also run all other slow tests from a set, either check or
check-qtest and that would add at least 2 minutest to the migration
run.
CI run: https://gitlab.com/farosas/qemu/-/pipelines/1499208871
Fabiano Rosas (4):
tests/qtest: Add check-migration
docs: Add migration tests documentation
tests/qtest/migration: Move tests into g_test_slow()
ci: Add check-migration-quick to the clang job
.gitlab-ci.d/buildtest.yml | 2 +-
docs/devel/testing/index.rst | 1 +
docs/devel/testing/main.rst | 13 ++
docs/devel/testing/migration.rst | 275 +++++++++++++++++++++++++++++++
docs/devel/testing/qtest.rst | 1 +
meson.build | 6 +-
tests/Makefile.include | 2 +
tests/qtest/meson.build | 47 +++++-
tests/qtest/migration-helpers.c | 9 +
tests/qtest/migration-helpers.h | 16 ++
tests/qtest/migration-test.c | 67 ++++----
11 files changed, 392 insertions(+), 47 deletions(-)
create mode 100644 docs/devel/testing/migration.rst
base-commit: 05adb38839ba656c7383a548b460d95c91e2febe
--
2.35.3
