Package: autopkgtests
Severity: minor
X-Debbugs-Cc: ni...@thykier.net
Hi
I have been using the text from README.package-tests.rst in `debputy`
for hover docs (to be shown to users). When I sent the hover docs texts
to review on debian-l10n-english, I got some feedback that applied
equally to the README.package-tests.rst file.
The quotes are from either
https://lists.debian.org/debian-l10n-english/2024/09/msg00008.html or
https://lists.debian.org/debian-l10n-english/2024/09/msg00009.html
Both emails are a general review and cover other parts. It might be
helpful to search for "Stanza:Tests" as that would bring you to texts
related to debian/tests/control.
## For `breaks-testbed`:
The text in question:
When this restriction is present the test will usually be skipped
unless the testbed's virtualisation arrangements are sufficiently
powerful, or alternatively if the user explicitly requests.
Remark from Richard:
It would be really helpful to say what sbuild does here, eg i think it
mostly skips such tests unless you use lxc?
I agree with Richard here in the sense that the documentation could be
more explicit on what is "sufficiently powerful" by means of examples.
## For `flaky`:
The test is expected to fail intermittently, and is not suitable for
gating continuous integration. This indicates a bug in either the
package under test, a dependency or the test itself, but such bugs
can be difficult to fix, and it is often difficult to know when the
bug has been fixed without running the test for a while. If a
``flaky`` test succeeds, it will be treated like any other
successful test, but if it fails it will be treated as though it
had been skipped.
Remark from Richard:
what on earth does this mean?
(With this referring to "gating continuous integration" being
highlighted)
Justin added:
A piece of continually grating jargonisation, but basically
https://devops.stackexchange.com/questions/10600/what-is-a-gating-continuous-integration-ci-system#10632
a test that must be passed before the new version is allowed in.
I agree with Richard and Justin that the jargon implies a bit too much
knowledge here. On the `debputy` side, I will have the jargon link to
the stackexchange page that Justin provided as a temporary measure. But
I think autopkgtests could do with an improvement here.
## For `isolation-container`
The `debputy` IDE editor support (`debputy-lsp`), `debputy` can provide
completion suggestions with one line of documentation. For the
`isolation-container` I proposed the synopsis for the restriction:
Container level isolation required
Richard responded with:
Can you say what this needs - chroot? schroot? lxc? docker? quemu?
it's really hard to understand from other docs so this would be a
great benefit to users
The thing is, I do not know when container level isolation is required.
Therefore, I have a hard time coming up with a better synopsis for this
restriction. I am hoping you can help me here.
The synopsis is shown when `debputy` provides `isolation-container` and
should be sufficient for the user to know whether this is the value they
want to complete. As mentioned, it is at most one line and limited in
space (soft limit; longer text may be truncated depending on
editor/font-size).
## For `Depends` (`@recommends`)
``@recommends@`` stands for all the packages listed in the
``Recommends:`` fields of all the binary packages mentioned in the
``debian/control`` file. Please note that variables are stripped,
so if some required test dependencies aren't explicitly mentioned,
they may not be installed.
Richard asked:
dont understand "stripped" in this para
I have to agree with Richard here. A binary package (.deb) never has
substitution variables and the text is phrased as if autopkgtests reads
the Recommends from the .deb (or a Packages file, but that is basically
the same as far as substitution variables go). So what exactly is being
stripped here?
## For `Depends`
If no Depends field is present, ``Depends: @`` is assumed. Note that
the source tree's Build-Dependencies are *not* necessarily
installed, and if you specify any Depends, no binary packages from
the source are installed unless explicitly requested.
Richard asked:
or requested via @?
I understood that as being a suggestion for:
If no Depends field is present, ``Depends: @`` is assumed. Note that
the source tree's Build-Dependencies are *not* necessarily
installed, and if you specify any Depends, no binary packages from
the source are installed unless explicitly requested or requested
via @.
## For `Tests`:
This field names the tests which are defined by this stanza, and map
to executables/scripts in the test directory
Here, the suggestion is to drop `/scripts`. The point is that it is an
executable and a script is an executable. The `/scripts` ends up being
noise.
See the mails linked above if you want more context for the suggestion.
Best regards,
Niels
