Hi Paul, On Thu, Nov 09, 2023 at 02:16:47PM +0100, Paul Gevers wrote: > I looked at the results of the autopkgtest of your package. I noticed that > it regularly fails (in this case because it's blocking migration of > src:autopkgtest), mostly on armhf and a bit on ppc64el and s390x. > > Because the unstable-to-testing migration software now blocks on > regressions in testing, flaky tests, i.e. tests that flip between > passing and failing without changes to the list of installed packages, > are causing people unrelated to your package to spend time on these > tests.
The fails don't look regular to me. Most of the times they are passing. It's difficult to figure out why it sometimes doesn't draw anything on the weaker architectures (the screenshot that is used for OCR is blank). I'm going to mark the test as flaky now. > PS: why does it even use text from a different and very unrelated package? > If there's not enough text in it's own source, couldn't it use something > that's installed on all Debian systems, such that it doesn't need to be > installed additionally and trigger migration runs? The autopkgtest documentation is available in HTML and plaintext (rst) format. That's needed to compare the OCR'd HTML-text against. It was the first example I could find, and I thought as autopkgtest is installed on systems running the tests, I could use it. If you are aware of other packages that are installed by default that ship documentation in HTML and some plaintext, then I could switch to it in the test. Kind regards, Reiner
