On Thu, Jul 17, 2025 at 1:04 PM Alex Bennée <alex.ben...@linaro.org> wrote: > > Manos Pitsidianakis <manos.pitsidiana...@linaro.org> writes: > > > On Thu, Jul 17, 2025 at 11:42 AM Alex Bennée <alex.ben...@linaro.org> wrote: > >> > >> Manos Pitsidianakis <manos.pitsidiana...@linaro.org> writes: > >> > >> > Add argument parsing to functional tests to improve developer experience > >> > when running individual tests. All logs are printed to stdout > >> > interspersed with TAP output. > >> > > >> > ./pyvenv/bin/python3 ../tests/functional/test_aarch64_virt.py --help > >> > usage: test_aarch64_virt [-h] [-d] > >> > >> Am I holding it wrong? > >> > >> ➜ ./pyvenv/bin/python ../../tests/functional/test_aarch64_virt.py --help > >> Traceback (most recent call last): > >> File > >> "/home/alex/lsrc/qemu.git/builds/all/../../tests/functional/test_aarch64_virt.py", > >> line 16, in <module> > >> from qemu_test import QemuSystemTest, Asset, > >> exec_command_and_wait_for_pattern > >> File > >> "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/__init__.py", line > >> 14, in <module> > >> from .testcase import QemuBaseTest, QemuUserTest, QemuSystemTest > >> File > >> "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", line > >> 26, in <module> > >> from qemu.machine import QEMUMachine > >> ModuleNotFoundError: No module named 'qemu' > >> > >> I thought the point of the venv is we had all the modules we need > >> automatically available to the PYTHONPATH? > > > > Is PYTHONPATH exported? Check that you've done the instructions > > detailed here: > > https://www.qemu.org/docs/master/devel/testing/functional.html#running-tests > > We should probably include enough in the commit message so a blind copy > and paste works, or at least indicates you need more setup to run the > test: > > set -x PYTHONPATH ../../python:../../tests/functional/ > env QEMU_TEST_QEMU_BINARY=./qemu-system-aarch64 ./pyvenv/bin/python > ../../tests/functional/test_aarch64_virt.py --debug > > >> > >> > > >> > QEMU Functional test > >> > > >> > options: > >> > -h, --help show this help message and exit > >> > -d, --debug Also print test and console logs on stdout. This will > >> > make the TAP output invalid and is meant for debugging > >> > only. > >> > > >> > Signed-off-by: Manos Pitsidianakis <manos.pitsidiana...@linaro.org> > >> > --- > >> > docs/devel/testing/functional.rst | 2 ++ > >> > tests/functional/qemu_test/testcase.py | 51 > >> > ++++++++++++++++++++++++++++++++-- > >> > 2 files changed, 50 insertions(+), 3 deletions(-) > >> > > >> > diff --git a/docs/devel/testing/functional.rst > >> > b/docs/devel/testing/functional.rst > >> > index > >> > 9e56dd1b1189216b9b4aede00174c15203f38b41..9d08abe2848277d635befb0296f578cfaa4bd66d > >> > 100644 > >> > --- a/docs/devel/testing/functional.rst > >> > +++ b/docs/devel/testing/functional.rst > >> > @@ -63,6 +63,8 @@ directory should be your build folder. For example:: > >> > $ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64 > >> > $ pyvenv/bin/python3 ../tests/functional/test_file.py > >> > > >> > +By default, functional tests redirect informational logs and console > >> > output to > >> > +log files. Specify the ``--debug`` flag to also print those to standard > >> > output. > >> > The test framework will automatically purge any scratch files created > >> > during > >> > the tests. If needing to debug a failed test, it is possible to keep > >> > these > >> > files around on disk by setting ```QEMU_TEST_KEEP_SCRATCH=1``` as an env > >> > diff --git a/tests/functional/qemu_test/testcase.py > >> > b/tests/functional/qemu_test/testcase.py > >> > index > >> > 2082c6fce43b0544d4e4258cd4155f555ed30cd4..fad7a946c6677e9ef5c42b8f77187ba836c11aeb > >> > 100644 > >> > --- a/tests/functional/qemu_test/testcase.py > >> > +++ b/tests/functional/qemu_test/testcase.py > >> > @@ -11,6 +11,7 @@ > >> > # This work is licensed under the terms of the GNU GPL, version 2 or > >> > # later. See the COPYING file in the top-level directory. > >> > > >> > +import argparse > >> > import logging > >> > import os > >> > from pathlib import Path > >> > @@ -31,6 +32,20 @@ > >> > from .uncompress import uncompress > >> > > >> > > >> > +def parse_args(test_name: str) -> argparse.Namespace: > >> > + parser = argparse.ArgumentParser( > >> > + prog=test_name, description="QEMU Functional test" > >> > + ) > >> > + parser.add_argument( > >> > + "-d", > >> > + "--debug", > >> > + action="store_true", > >> > + help="Also print test and console logs on stdout. This will > >> > make the" > >> > + " TAP output invalid and is meant for debugging only.", > >> > + ) > >> > + return parser.parse_args() > >> > + > >> > + > > I'm definitely onboard for improving the ergonomics of calling the tests > directly. Others to consider: > > - triggering the behaviour of QEMU_TEST_KEEP_SCRATCH > - dumping the command line without the test harness QMP/serial pipes > connected
Definitely useful, I've needed those behaviors in the past! I will send more patches. I will also see if it's easy to make it possible to list the tests that would run in a specific file, and if possible to run only specified tests. Thanks, -- Manos Pitsidianakis Emulation and Virtualization Engineer at Linaro Ltd
