On Sat, 16 Jul 2022 20:55:02 +0200 Lucas Nussbaum <lu...@debian.org> wrote:
> Hi,
>
> The script I use to extract the failure from the log might have guessed
> badly indeed, however the build still fails, as shown by the full log
> linked in the bug:
> http://qa-logs.debian.net/2022/07/16/meson_0.63.0-1_unstable.log
> SO I wouldn't call that a "spurious failure".
>
> However looking at the log I don't really understand why the
> override_dh_auto_test target fails. Can you explain? Maybe I can improve
> my script.
>
> Lucas
The testsuite has two parts. The first part (that actually fails in this
case) is a standard python-unittest run, and presumably you have
heuristics that can catch that.
The second part is a very custom setup, you'll undoubtedly need to
manually code support for it. Since Debian includes the environment
variable $MESON_PRINT_TEST_OUTPUT for extra extra extra debug spew, it
logs full configure/build/test/install output for all tests, not just
the ones that represent test failures... including, as stated above,
passing tests that attempt to assert that certain things aren't expected
to work.
Let's take a look at run_project_tests.py which is driving these very
verbose tests:
class TestStatus(Enum):
OK = normal_green(' [SUCCESS] ')
SKIP = yellow(' [SKIPPED] ')
ERROR = red(' [ERROR] ')
UNEXSKIP = red('[UNEXSKIP] ')
UNEXRUN = red(' [UNEXRUN] ')
CANCELED = cyan('[CANCELED] ')
RUNNING = blue(' [RUNNING] ') # Should never be actually printed
LOG = bold(' [LOG] ') # Should never be actually printed
Individual test logs are always separated by lines beginning with the
string " [XXX]".
error, unexskip, and unexrun are the only failure states.
unexskip and unexrun can only happen in Meson's internal CI
($MESON_CI_JOBNAME is set and isn't "thirdparty").
So you can just filter for:
- any python-unittest matching logs (e.g. in between "===========" and
"----\nRan <XXX> tests in <XXX.XXX>s")
- anything in between " [ERROR]" and " [<success/skipped>]"
--
Eli Schwartz
