On Thu, Apr 23, 2020 at 08:14:59AM -0700, Mark Janes wrote:
Danylo Piliaiev <danylo.pilia...@gmail.com> writes:I want to sum up what happened from my perspective, I think it could be useful to improve the process: 1) There was a regression in 20.* (https://gitlab.freedesktop.org/mesa/mesa/-/issues/2758) 2) I "fixed" the regression and broke non-iris drivers (https://gitlab.freedesktop.org/mesa/mesa/-/commit/d684fb37bfbc47d098158cb03c0672119a4469fe) 3) I "fixed" the new regression of fix 2) (https://gitlab.freedesktop.org/mesa/mesa/-/commit/829013d0cad0fa2513b32ae07cf8d745f6e5c62d) 4) After that, it appeared that due to a bug in piglit, Intel CI didn't run piglit tests which gave me a false sense of commit's correctness (https://gitlab.freedesktop.org/mesa/mesa/-/issues/2815) 5) I aimed to have a fix before the next minor release on 2020-04-29 by consulting the release calendar. 6) I accidentally saw 20.0.5 being released with one of the two of my commits. I see multiple failure points: 1) Me not carefully examining all code paths, because at least one that failed should have been obvious to test.You missed one important failure point: 1a) untested piglit commits pushed to master which disabled the test suite. I was impressed by how quickly regressions were added to master in the few days that piglit was disabled in CI. At least 4 regressions in as many days.2) CI not communicating that piglit tests were not executed. Again, I could have seen this, examined CI results, but did not.This was my fault. Over 5 years ago, I change the CI harness to ignore errors thrown by piglit when it is run on an empty test set. There are valid CI use cases (when bisecting) where the CI will attempt to run a small test set on older platforms that do not support any of the tests. A more defensive implementation would check that an empty test set is allowed only in the specific/expected use case.
I fixed the (intel) CI by implementing a check for this (and thanks Dylan for the piglit patch to help facilitate this check), all other piglit failures should be immediate and visible from now on. -Clayton
signature.asc
Description: PGP signature
_______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
