Here’s what Engineering Productivity has accomplished in Q3. In Q4, we’re switching to OKR’s for tracking goals and progress; these should be published soon.
== Build System == Overall goal: Reduce build times for local developers and automation; improve maintainability. Q3 progress: * TaskCluster tasks now running on faster AWS instances. Execution time decreases of several minutes across the board. * Initial support for building with Tup build system. * Libffi’s configure script, formerly a long pole when running configure, is no longer run, and its functionality has been incorporated into our Python configure infrastructure. Additionally, libffi no longer builds with its own build system and is now incorporated into moz.build. * Significant progress has been made to convert our autoconf based configure script to a more performant and maintainable Python based system. The quantity of .m4 and .sh code invoked during this part of the build has been reduced by ⅓. == MozReview == Overall goal: Increase adoption of MozReview for mozilla-central code reviews Q3 progress: * New, custom theme * “Finish Review” dialog reworked for improved usability * Automatic LDAP association * Display diff statistics in MozReview and BMO * Prototyped review-request UI changes to improve clarity and discoverability * New Vagrant-based development environment to encourage contributions == Debugging UX == Overall goal: Make it easier to debug and resolve automated test failures Project page: https://wiki.mozilla.org/EngineeringProductivity/Projects/Debugging_UX_improvements Sub-goal: Created simple workflow to run test suites from an interactive (one-click) loaner Q3 progress: * Deployed automation changes that provide developers with the ability to run tests, clone mozilla-central, and obtain shell access on actual linux64 test workers with just a few clicks from Treeherder. Sub-goal: Reduce median end-to-end times on Try to 60 minutes Q3 progress: * Download zip files into memory before unzipping to save writing the zip file to disk first * Using ‘--artifact’ flag in try syntax replaces all scheduled build jobs with an opt artifact build on linux, linux64, mac64, win32, win64. |./mach try| also recognizes --artifact flag and prevents the user from scheduling compiled-code tests with it. Compiled-code test jobs report an ‘exception’ result when run against an artifact build on try. * Made progress on rewriting sccache in Rust, after which we can deploy a two-tier sccache for Try builds, that will allow Try builds to consume sccache objects from mozilla-central * Build jobs in automation are now using version control clone+checkout best practices; this results in faster VCS operations, less overhead during automation jobs * WPT tests now running from source checkout * Work in progress to show high level “end-to-end times” across all platforms and tests, on Try. This is building on previous work, which is a prototype limited to build times: http://people.mozilla.org/~klahnakoski/MoBuildbotTimings/Builds-Overview.html == Other Projects == Bugzilla * Memory leak stuff previously discussed was implemented this quarter * BMO implemented CSP for some pages, yielding an A- on Mozilla Observatory * BMO switched to vendored dependencies which will result in a much faster development pace * Legacy bzapi saw a performance improve 226s to 184ms or 122826% faster ( http://dylanwh.tumblr.com/post/151314085067/sorry-i-meant-i-changed-it-from-226s-to-184ms ) * Work continues on search speed improvements; when landed, keyword search for intermittent-failure goes from 2 to 0.08s Treeherder * Migrated Treeherder from SCL3 to Heroku. * Use ReactJS to make the exclusion editor (Sheriff panel / Admin) usable * Support submitting Github resultsets via Pulse Continuous Integration and Test Harnesses * Stood up flake8 lint job with mozlint, and converted eslint job to use mozlint * Finished migrating majority of jobs from windows hardware to windows VM in AWS, only a few remain running on HW with no plans to move to windows vm VM to limitations of the VM’s * Implemented Edge support in wptrunner * Did initial exploratory work on cross-browser comparisons using web-platform-tests and wptrunner * Our Outreachy intern, Anjana Vakil, landed log parsing for marionette-harness tests on Treeherder by integrating a pytest-mozlog plugin into mozlog. Marionette-harness tests are thus ready to be Tier-1 on Treeherder. * Documented some best practices for code backouts; see https://wiki.mozilla.org/Sheriffing/How:To:Backouts#Best_Practices_and_Communication Janitor (https://janitor.technology) * Refactored the Janitor service into a multi-server cluster (multiple Docker hosts with Node.js routing agent, TLS / OAuth2 token-based communication) * Completed preliminary service review, started hardening (formalized + implemented firewall rules, started implementing Mozilla’s security guidelines, scheduled rapid risk assessment) W3C WebDriver and Marionette * Released geckodriver 0.10 with multiple fixes including support for configuring Firefox through capabilities * Extensive work on geckodriver 0.11 with improved configuration support, win32 support, better logging * Added webdriver spec tests to treeherder (on linux64). * Wrote specification for “actions” support in WebDriver, and Marionette implementation is in review * Deployed Marionette tests on Fennec in TaskCluster; they are Tier-3 on Treeherder as we make progress with greening up the tests. Code Coverage * Fall session of UCOSP has started! There are three students, plus a student mentor, continuing work from last year; this time adding C++ code coverage to our tests so it is easy to trigger with a mach command. Meeting notes are here: https://public.etherpad-mozilla.org/p/code_coverage_f16 * UCOSP is a program that brings together students from across Canada to work together on open source projects. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
