Hello,

As described in https://bugreports.qt.io/browse/QTQAINFRA-6592, we currently 
execute ctest -V (verbose), which logs all of the output from all tests to the 
resulting test log, even for passing tests.


This practice presents challenges in investigating test failures, as it 
requires loading and navigating through approximately 25MB (qtbase) of logs 
within a (mobile) web browser or editor to identify a test failure.


To address this issue, we propose implementing 
https://codereview.qt-project.org/c/qt/qtbase/+/592470, which will enable us to 
pass —output-on-failure to ctest. This modification will significantly reduce 
the ctest output and signal-to-noise ratio.


In certain cases, it may still be beneficial to examine the full verbose log. 
Therefore, we will continue to store this information as CTest.log within the 
testresults.tar.gz for those who require a comprehensive analysis.


It is important to note that enabling —output-on-failure will not impact our 
test reporting to https://testresults.qt.io/grafana. This reporting is based on 
XML output that we still generate for each test.


An example of how this change would manifest is available here: 
https://testresults.qt.io/coin/integration/qt/qtbase/tasks/1727203984


Let us know if you have any concerns not addressed in the corresponding bug 
reports.


Cheers,

Tor Arne
-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

Reply via email to