First of all, sorry for the radio silence on our end. We decided to
rerun the tests and forgot to reply here that we were doing so.

As discussed out-of-band: for future SRU verification, we plan on adding
specific output to our integration tests that say which version is being
tested.

For this release, this is what we've done:

We tested and used this feature to run all of the integration tests with
proposed enabled: https://github.com/canonical/ubuntu-advantage-
client/pull/1673

You can verify that this was used in the new output attached because
"enable_proposed = True" is logged in the "Config options" section at
the top of each file.

To further ensure that not only was proposed enabled, but also that the
correct version was tested, we ran additional ad-hoc tests using the
enable_proposed feature. These additional tests have extra logging that
does output the version tested by running `ua version` on the machine.
These are named "behave-lxd-YY.MM-ua-version.txt" in the attached tar.
The extra logging looks like this:

2021-06-17 10:36:07,624 - ua-ci - INFO - Validating ubuntu-advantage-
tools v.27.1~20.04.1

We believe that this is sufficient evidence to say that our
"enable_proposed" feature is using the correct version of ubuntu-
advantage-tools for all of our integration tests. As a result, I'm
changing the tags back to verification-done.

We recognize that this is not a great solution and plan on improving it
for our next release.

** Attachment added: "ua-27.1-proposed-with-version-evidence.tar.xz"
   
https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1929597/+attachment/5505336/+files/ua-27.1-proposed-with-version-evidence.tar.xz

** Tags removed: verification-needed verification-needed-bionic 
verification-needed-focal verification-needed-groovy 
verification-needed-hirsute verification-needed-xenial
** Tags added: verification-done verification-done-bionic 
verification-done-focal verification-done-groovy verification-done-hirsute 
verification-done-xenial

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1929597

Title:
  [SRU] ubuntu-advantage-tools (27.0.2 -> 27.1) Xenial, Bionic, Focal,
  Groovy, Hirsute

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ubuntu-advantage-tools/+bug/1929597/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to