https://lists.ubuntu.com/archives/kernel-team/2020-May/109742.html
** Description changed: - There are some test cases in ftrace from ubuntu_kernel_selftests that - would need some kernel configs to be enabled. + == SRU Justification == + There are some test cases in ftrace from ubuntu_kernel_selftests that would need some kernel configs to be enabled or have some special HW requirement. + + https://pad.lv/1869347 - missing CONFIG_SAMPLE_TRACE_PRINTK=m + https://pad.lv/1869349 - missing CONFIG_SAMPLE_TRACE_PRINTK=m + https://pad.lv/1874198 - nproc > 1 + + If the requirement was not met, these test will fail with "unresolved" and return 1. So even these is no failed test case, the ftrace test will still be marked as "failed" with the presence of unresolved test case: + # # of passed: 28 + # # of failed: 0 + # # of unresolved: 1 + # # of untested: 0 + # # of unsupported: 58 + # # of xfailed: 0 + # # of undefined(test bug): 0 + not ok 1 selftests: ftrace: ftracetest # exit=1 + + == Fix == + * b730d668138c ("ftrace/selftest: make unresolved cases cause failure if --fail-unresolved set") + + This patch will only make unresolved test case return 1 if we ask it to + fail with unresolved test case. + + It can be cherry-picked to kernels starting from Disco. + For Bionic and Xenial it needs to be backported with some context adjustment as it's missing some other patches to the testing tools. + + == Test == + With the patch applied, the ftrace test will not be marked as failed with the presence of unresolved test case: + + # # of passed: 28 + # # of failed: 0 + # # of unresolved: 1 + # # of untested: 0 + # # of unsupported: 58 + # # of xfailed: 0 + # # of undefined(test bug): 0 + ok 1 selftests: ftrace: ftracetest + + + == Regression Potential == + Low, changes limited to testing tools, no actual impacts to our kernels. + + The worst case is to cause false negative in test reports, but in that + case it's probably a test case issue (a failed case should be treated as + a failed one, instead of unresolved) + + + == Original Bug Report == + There are some test cases in ftrace from ubuntu_kernel_selftests that would need some kernel configs to be enabled. bug 1869347 - bug 1869349 bug 1869349 We should not treat unresolved test case as a failure, as it's expected to fail with no correlated configs enabeled. https://github.com/torvalds/linux/commit/b730d668138cb3dd9ce78f8003986d1adae5523a #diff-33d74da785cbc5da20f71eb1af106d68 ** Description changed: == SRU Justification == - There are some test cases in ftrace from ubuntu_kernel_selftests that would need some kernel configs to be enabled or have some special HW requirement. + There are some test cases in ftrace from ubuntu_kernel_selftests that + would need some kernel configs to be enabled or have some special HW + requirements: + * https://pad.lv/1869347 - missing CONFIG_SAMPLE_TRACE_PRINTK=m + * https://pad.lv/1869349 - missing CONFIG_SAMPLE_TRACE_PRINTK=m + * https://pad.lv/1874198 - nproc > 1 - https://pad.lv/1869347 - missing CONFIG_SAMPLE_TRACE_PRINTK=m - https://pad.lv/1869349 - missing CONFIG_SAMPLE_TRACE_PRINTK=m - https://pad.lv/1874198 - nproc > 1 - - If the requirement was not met, these test will fail with "unresolved" and return 1. So even these is no failed test case, the ftrace test will still be marked as "failed" with the presence of unresolved test case: + If the requirement was not met, these test will fail with "unresolved" + and return 1. So even these is no failed test case, the ftrace test + will still be marked as "failed" with the presence of unresolved test: # # of passed: 28 # # of failed: 0 # # of unresolved: 1 # # of untested: 0 # # of unsupported: 58 # # of xfailed: 0 # # of undefined(test bug): 0 not ok 1 selftests: ftrace: ftracetest # exit=1 == Fix == * b730d668138c ("ftrace/selftest: make unresolved cases cause failure if --fail-unresolved set") - This patch will only make unresolved test case return 1 if we ask it to - fail with unresolved test case. + This patch will only make unresolved test case return 1 if we ask it + to fail with unresolved test case. It can be cherry-picked to kernels starting from Disco. - For Bionic and Xenial it needs to be backported with some context adjustment as it's missing some other patches to the testing tools. + For Bionic and Xenial it needs to be backported with some context + adjustment as they're missing some other patches to the testing tool. == Test == - With the patch applied, the ftrace test will not be marked as failed with the presence of unresolved test case: - - # # of passed: 28 - # # of failed: 0 - # # of unresolved: 1 - # # of untested: 0 - # # of unsupported: 58 - # # of xfailed: 0 - # # of undefined(test bug): 0 - ok 1 selftests: ftrace: ftracetest - + With the patch applied, the ftrace test will not be marked as failed + with the presence of unresolved test case: + # # of passed: 28 + # # of failed: 0 + # # of unresolved: 1 + # # of untested: 0 + # # of unsupported: 58 + # # of xfailed: 0 + # # of undefined(test bug): 0 + ok 1 selftests: ftrace: ftracetest == Regression Potential == Low, changes limited to testing tools, no actual impacts to our kernels. The worst case is to cause false negative in test reports, but in that - case it's probably a test case issue (a failed case should be treated as - a failed one, instead of unresolved) + case it's probably a test case issue since a failed case should be + treated as a failed one, instead of unresolved. == Original Bug Report == There are some test cases in ftrace from ubuntu_kernel_selftests that would need some kernel configs to be enabled. bug 1869347 bug 1869349 We should not treat unresolved test case as a failure, as it's expected to fail with no correlated configs enabeled. https://github.com/torvalds/linux/commit/b730d668138cb3dd9ce78f8003986d1adae5523a #diff-33d74da785cbc5da20f71eb1af106d68 ** Tags added: bionic disco eoan focal xenial -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1877958 Title: Do not treat unresolved test case in ftrace from ubuntu_kernel_selftests as failure Status in ubuntu-kernel-tests: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Xenial: In Progress Status in linux source package in Bionic: In Progress Status in linux source package in Eoan: In Progress Status in linux source package in Focal: In Progress Bug description: == SRU Justification == There are some test cases in ftrace from ubuntu_kernel_selftests that would need some kernel configs to be enabled or have some special HW requirements: * https://pad.lv/1869347 - missing CONFIG_SAMPLE_TRACE_PRINTK=m * https://pad.lv/1869349 - missing CONFIG_SAMPLE_TRACE_PRINTK=m * https://pad.lv/1874198 - nproc > 1 If the requirement was not met, these test will fail with "unresolved" and return 1. So even these is no failed test case, the ftrace test will still be marked as "failed" with the presence of unresolved test: # # of passed: 28 # # of failed: 0 # # of unresolved: 1 # # of untested: 0 # # of unsupported: 58 # # of xfailed: 0 # # of undefined(test bug): 0 not ok 1 selftests: ftrace: ftracetest # exit=1 == Fix == * b730d668138c ("ftrace/selftest: make unresolved cases cause failure if --fail-unresolved set") This patch will only make unresolved test case return 1 if we ask it to fail with unresolved test case. It can be cherry-picked to kernels starting from Disco. For Bionic and Xenial it needs to be backported with some context adjustment as they're missing some other patches to the testing tool. == Test == With the patch applied, the ftrace test will not be marked as failed with the presence of unresolved test case: # # of passed: 28 # # of failed: 0 # # of unresolved: 1 # # of untested: 0 # # of unsupported: 58 # # of xfailed: 0 # # of undefined(test bug): 0 ok 1 selftests: ftrace: ftracetest == Regression Potential == Low, changes limited to testing tools, no actual impacts to our kernels. The worst case is to cause false negative in test reports, but in that case it's probably a test case issue since a failed case should be treated as a failed one, instead of unresolved. == Original Bug Report == There are some test cases in ftrace from ubuntu_kernel_selftests that would need some kernel configs to be enabled. bug 1869347 bug 1869349 We should not treat unresolved test case as a failure, as it's expected to fail with no correlated configs enabeled. https://github.com/torvalds/linux/commit/b730d668138cb3dd9ce78f8003986d1adae5523a #diff-33d74da785cbc5da20f71eb1af106d68 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1877958/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
