Public bug reported:
r1920
Today I mistakingly pushed a new unit test in an MR without "bzr add"ing
the corresponding QML file.
CI didn't say anything about it, tests were "PASS" on all architectures, the MR
was approved, and it was about to land. Later in the day I realized I had not
pushed the QML file.
But still, I got all PASS when I should have clearly got FAIL.
So I manually checked the (PASSing) console log from Jenkins, and found:
- a QWARN in the test saying that it couldn't find the QML file
- a segfault in the same test, due to the previous point
The UITK test suite has a script that runs the tests and checks, beside
other things, that there were no QWARN issues by tests. That check is
done by parsing a per-test result .xml that is written at the end of
each test.
I think what is happening in this case is that the .xml file is not
being created at all because of the segfault, and the test runner is not
detecting that.
So, potentially, there are other segfaults in other test results that
have never been caught so far, and could uncover new test failures :)
** Affects: ubuntu-ui-toolkit (Ubuntu)
Importance: Critical
Assignee: Christian Dywan (kalikiana)
Status: Confirmed
** Changed in: ubuntu-ui-toolkit (Ubuntu)
Assignee: (unassigned) => Christian Dywan (kalikiana)
** Changed in: ubuntu-ui-toolkit (Ubuntu)
Importance: Undecided => Critical
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to ubuntu-ui-toolkit in
Ubuntu.
https://bugs.launchpad.net/bugs/1567286
Title:
Test runner script gives PASS on segmentation faults
Status in ubuntu-ui-toolkit package in Ubuntu:
Confirmed
Bug description:
r1920
Today I mistakingly pushed a new unit test in an MR without "bzr
add"ing the corresponding QML file.
CI didn't say anything about it, tests were "PASS" on all architectures, the
MR was approved, and it was about to land. Later in the day I realized I had
not pushed the QML file.
But still, I got all PASS when I should have clearly got FAIL.
So I manually checked the (PASSing) console log from Jenkins, and found:
- a QWARN in the test saying that it couldn't find the QML file
- a segfault in the same test, due to the previous point
The UITK test suite has a script that runs the tests and checks,
beside other things, that there were no QWARN issues by tests. That
check is done by parsing a per-test result .xml that is written at the
end of each test.
I think what is happening in this case is that the .xml file is not
being created at all because of the segfault, and the test runner is
not detecting that.
So, potentially, there are other segfaults in other test results that
have never been caught so far, and could uncover new test failures :)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-ui-toolkit/+bug/1567286/+subscriptions
--
Mailing list: https://launchpad.net/~touch-packages
Post to : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help : https://help.launchpad.net/ListHelp
