On Tue, Sep 13, 2016 at 1:02 PM, Dustin Mitchell <dus...@mozilla.com> wrote:
> I think many of the concerned people were flagged for r? on bug > 1286075, but for everyone else, it's landed in inbound today. > > If you haven't mucked around with the configuration of which > TaskCluster tasks run for pushes, and how they run, then you can tune > out now. > > If you have a project that I've rudely said "please wait until I land > this", then now is your chance! > > The changes were pretty wide-spread, but the upshot is this: > taskcluster/ci/legacy is gone, as as all of the yaml files with their > weird inheritance hierarchies. In their place is a better > implementation that should be more semantic and easier to modify or > extend. This is well documented under http://gecko.readthedocs.io/ > (or will be once it lands in central). > > Please send any questions my way, as always! > Dustin undersells the importance of his work. So let me try. Essentially what Dustin did was rewrite the in-tree automation scheduling code to make it simpler and easier to hack on. It is kinda like going from Makefile.in files to moz.build files. You can now drop a .yml file into a directory and TaskCluster will start running automation for it and the results will show up in Treeherder. For example, to add a new linter, simply add an entry to https://hg.mozilla.org/integration/mozilla-inbound/file/82d0a583a9a39bf0b0000bccbf6d5c9ec2596bcc/taskcluster/ci/source-check/mozlint.yml. You don't need any help from anyone to add new tasks: just add a few lines of YAML and it runs. We had this before. However, there was a pile of inheritance and spaghetti code. With the rewrite, each job/task type (build, desktop test, linter, etc) has a well-defined type and behavior and the Python code that converts it from YAML to what automation schedules is pretty easy to follow and debug. It abstracts many of the complexities away from YAML so people who only care about defining new tasks or tweaking existing ones can do so without as much head scratching. This work builds upon Dustin's earlier work of converting the in-tree scheduling code to use a graph data structure to represent tasks. This allows us to do crazy things like say define a task to build the Rust toolchain that is a dependency of the build task so that any time we change how the Rust toolchain is built, we automatically build the Rust toolchain and feed its output into the build task. This makes deterministic and reproducible builds much easier to achieve. It also essentially makes TaskCluster a distributed build system. And that makes me giddy. Dustin's work and the overall work to convert things to TaskCluster are an incredible transformation from the days where you had to define buildbot configs on a server somewhere and couldn't easily add or experiment with new automation. Everything is in tree and developers are in almost complete control to modify [TaskCluster] automation to their heart's content. It is really quite amazing. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
