On Friday, January 25, 2019 11:08:52 AM CET Lars Knoll wrote: > To me this means we need to seriously rethink that part of our CI system, > and ideally test changes (or patch series) individually and in parallel. So > maybe we should adjust our CI system that way: > > * test changes (or patch series) individually > * If they pass CI merge or cherry-pick them into the branch > * reject only if the merge/cherry-pick gives conflicts > > This adds a very small risk that two parallel changes don’t conflict during > the merge/cherry-pick process, but cause a test regression together. To > help with that, we can simply run a regular status check on the repo. If > this happens the repo will be blocked for further testing until someone > submits a fix or reverts an offending change, which is acceptable. > > Another advantage is that this would pave the road towards a system where CI > testing happens before human review, so we can in the longer term avoid > duplicated review/approval work.
Hi, I do agree with it. Personally, I'm convinced that we are doing CI gate- keeping in a wrong way. CI is about balancing costs vs. gains. We can not test everything, we should find as many as possible bugs, while running in reasonable time without using too much resources. That is the function to optimize. We really should test things in parallel. The risk of conflicts / regression caused by parallel integration in project as big as qt is minimal. One can be even proactive and not stage changes while other similar ones are integrating. While broken repository state is bad, it is also easy to detect; nothing integrates because of one test, which is not known to be flaky (testrestults.qt.io/grafana...). Cheap (a.k.a reasonably fast to implement) proposal: - Stage button would create a stage branch as currently (through cherry-pick on top of target branch) - 3-5 minutes old stage branch would changed to be a build branche and all patches in it would be changed to "integrating" state. CI would start testing the build branch. At that time a new stage branch may be created. As opposite to now, multiple build branches may be tested in parallel. - Once a build branch testing passed we cherry-pick / merges (depends what gerrit does) changes on top of the target branch, in case of conflicts we mark it as failed. Advantages: - changes are tested in smaller batches => smaller chances to fail - we can use hardware in a more efficient way => more code integrated per hour - fairly easy to implement => we would not need to wait for it to happen years - that would be a step forward in to direction of testing before reviewing Disadvantages: - risk of breakages going through CI - someone work needs to be done to enable it Cheers, Jędrek _______________________________________________ Development mailing list [email protected] https://lists.qt-project.org/listinfo/development
