Thank you guys, this is a good starter.
On 01/11/2016 02:06 PM, Stef Walter wrote:
On 11.01.2016 12:08, Marius Vollmer wrote:
Jan Chaloupka <[email protected]> writes:
cockpit integration tests have proofed to be very useful for
discovering regression and bugs for kubernetes.
I am happy to hear that!
[...]
Would it be possible to run kubernetes related cockpit integration
tests on each new build of kubernetes in rawhide?
We have pretty much given up on running our tests on Fedora Rawhide.
- Testing "branched" seems much more relevant than testing "rawhide".
- Creating a VM image for rawhide seems to require a lot of luck.
- At the time we didn't have the resource to test evey pull request
against rawhide, and without that, we didn't seem to care enough
about the results to actually do something about it in a timely
manner.
That doesn't mean that you can't test rawhide kubernetes RPMs. You would
just test those RPMs against fedora-testing or fedora-23 instead.
Actually, testing rawhide build on f23 is better as I can see right away
if the build is a good candidate for f23 update.
This might all have changed, but once Fedora 24 is branched, I think
we'll put some effort into testing against that for every PR.
Do you really want to test against Rawhide, or are you happy to go along
with our choice of OSes? (Now with Debian! Come while it's green! :)
Fedora is preferred. If Debian can help to discover additional issues,
why not use both of them.
Once I tried to run the tests and ended up with downloading a lot of
images each over 1GB.
Yes, the copmplete set of tests needs quite some images initially, such
as complete FreeIPA and OpenShift installations. This can be reduced if
you only want to run a subset of the tests, but it's only an initial
setup cost so maybe it is better to just eat it once instead of spending
a few days on trying to save one hour of downloading. :)
Definitely. The Jenkins script must respect that and does not delete all
images before every run.
How hard it would be to write down a simple script that would expect
kubernetes build and distribution? As a result, it would set
everything up and run required tests (without user intervention). Or
if possible to use your system (possibly Jenkins?) and periodically
run integration tests (or with a suitable trigger).
I think with the recent addition of "vm-customize" this should not be
hard, not much harder than running the tests in the first place.
Something like this:
$ export TEST_OS=fedora-23
$ ./testsuite-prepare
$ ./vm-customize -i kubernetes-*.rpm
$ ./check-kubernetes
From there you should be able to quite easily add your own non-Cockpit
tests.
This is better than what I was recommending earlier. But you still need
to remember to run '$ sudo ./vm-prep'.
It looks so easy :).
Stef
_______________________________________________
cockpit-devel mailing list
[email protected]
https://lists.fedorahosted.org/admin/lists/[email protected]
_______________________________________________
cockpit-devel mailing list
[email protected]
https://lists.fedorahosted.org/admin/lists/[email protected]
