On 11.01.2016 12:08, Marius Vollmer wrote: > Jan Chaloupka <[email protected]> writes: > >> cockpit integration tests have proofed to be very useful for >> discovering regression and bugs for kubernetes. > > I am happy to hear that! > >> [...] >> >> Would it be possible to run kubernetes related cockpit integration >> tests on each new build of kubernetes in rawhide? > > We have pretty much given up on running our tests on Fedora Rawhide. > > - Testing "branched" seems much more relevant than testing "rawhide". > > - Creating a VM image for rawhide seems to require a lot of luck. > > - At the time we didn't have the resource to test evey pull request > against rawhide, and without that, we didn't seem to care enough > about the results to actually do something about it in a timely > manner.
That doesn't mean that you can't test rawhide kubernetes RPMs. You would just test those RPMs against fedora-testing or fedora-23 instead. > This might all have changed, but once Fedora 24 is branched, I think > we'll put some effort into testing against that for every PR. > > Do you really want to test against Rawhide, or are you happy to go along > with our choice of OSes? (Now with Debian! Come while it's green! :) > >> Once I tried to run the tests and ended up with downloading a lot of >> images each over 1GB. > > Yes, the copmplete set of tests needs quite some images initially, such > as complete FreeIPA and OpenShift installations. This can be reduced if > you only want to run a subset of the tests, but it's only an initial > setup cost so maybe it is better to just eat it once instead of spending > a few days on trying to save one hour of downloading. :) > >> How hard it would be to write down a simple script that would expect >> kubernetes build and distribution? As a result, it would set >> everything up and run required tests (without user intervention). Or >> if possible to use your system (possibly Jenkins?) and periodically >> run integration tests (or with a suitable trigger). > > I think with the recent addition of "vm-customize" this should not be > hard, not much harder than running the tests in the first place. > > Something like this: > > $ export TEST_OS=fedora-23 > $ ./testsuite-prepare > $ ./vm-customize -i kubernetes-*.rpm > $ ./check-kubernetes > > From there you should be able to quite easily add your own non-Cockpit > tests. This is better than what I was recommending earlier. But you still need to remember to run '$ sudo ./vm-prep'. Stef _______________________________________________ cockpit-devel mailing list [email protected] https://lists.fedorahosted.org/admin/lists/[email protected]
