Engineering Productivity made great progress on our top-level goals in Q2; keep reading for a description of the highlights. For a sneak peek of what we're doing in Q3, see this Google Doc: https://goo.gl/Z2YgQ2
Platform Operations Top-Level Projects '''Build Faster''' Overall goal: Reduce build times for local developers and automation; improve maintainability. Q2 progress: * Artifact Builds were rolled out to all desktop and android Firefox engineers * Reduced PGO Build times by over 2 hours. * The build system now lazily installs test files, saving build time and making mach commands for test execution faster. * A significant amount of logic has been removed from configure and Makefiles, resulting in better performance. This also paves the way for replacing gnu make with a newer build backend, like tup or bazel. * A distributed global cache is being made usable by local builds, which can reduce the amount of time needed to make C++ builds, especially if you have a fast internet connection and/or are located in an office. This work will finish up in Q3. '''MozReview''' Overall goal: Increase adoption of MozReview for gecko code reviews Q2 progress: * A decision was made to fork Review Board, which will make iteration faster, at the cost of potentially losing the ability to incorporate upstream improvements easily. * Autoland has been changed to land to a special 'autoland' integration branch, instead of mozilla-inbound. The autoland branch is merged periodically to mozilla-central, similar to mozilla-inbound. * MozReview now uses Bugzilla-style r?/r+/r- flags instead of "ship it". * Anyone can change the reviewers for any review request, similar to Bugzilla’s attachments. This will allow, for instance, delegating a review to someone else without having to involve the author. * An automated CI system was brought online, which should increase MozReview's development velocity. * Actions are being published to Pulse. '''TaskCluster Migration''' Overall goal: Help the TaskCluster team migrate continuous integration from buildbot to TaskCluster Q2 progress: * Most linux-based builds and tests are now running in TaskCluster, as Tier 1 or Tier 2, with the exception of linux64 Talos tests. For Talos, an experiment was conducted to determine whether the tests could be moved to AWS or to a docker-based hardware solution, but tests results in either case are too variable, so Talos will be ported to TaskCluster on native hardware at a later date. * Considerable progress has been made in porting Windows unit tests to AWS; this frees up Windows hardware capacity (reducing wait times, especially on Try), and paves the way for migration of Windows tests to TaskCluster. Other Projects '''Treeherder''' * A new bug filer tool was created for filing intermittent bug failures. * Treeherder now ingests test results from TaskCluster using Pulse; this allows those test results to be visible in staging and dev environments, as well as production. * Adding new TaskCluster-based jobs to a push in Treeherder is now possible. * The first version of automatic classification has been deployed; this automatically classifies some known intermittents as such, without the involvement of a sheriff. Work is underway to expand the number of failure types that auto classification recognizes. '''Performance Testing''' * Some improvements were made to make it easier to display non-Talos performance metrics in Perfherder. For example, Servo is now using Perfherder, see: https://mzl.la/29JyMMD * Support has been added for micro benchmarks, see e.g., https://mzl.la/29AkESh * System utilization during unit tests is now being reported to Perfherder. '''Mobile Testing''' * Autophone (Android phone-based automation) is now reporting to Treeherder as Tier 2. It now has a new dashboard: http://phonedash.mozilla.org '''Bugzilla''' * A major memory leak was fixed, leading to much longer lifetimes for the webhead processes, and a small performance gain. * You can now view and comment on GitHub pull-request diffs in Splinter. * Many modal-UI bugs fixed. * New textual bug-summary field. * The production database is now in UTC. * Cisco Spark support. * Many bugs in the in-progress upstream-merge branch (merging in recent upstream Bugzilla work) have been fixed. '''Marionette/WebDriver''' * Support for running WebDriver tests in gecko and Servo has been added to web-platform-tests. * Shipped geckodriver (https://github.com/mozilla/geckodriver), the HTTPD frontend to Marionette, and gathered user feedback * Aligned a number of commands to the WebDriver specification. * Marionette support has been added to Fennec. * Unit tests of the Marionette test harness have been expanded. '''Firefox UI Tests''' * Firefox UI Tests was the project of the month in June: http://bit.ly/29JBamC * With esr38 being deprecated, mozmill tests are now retired, and all Firefox UI Tests are Marionette-based and live in-tree. * Functional tests for linux have been moved to TaskCluster. * Test results in Treeherder have been promoted to Tier 2. '''Version Control''' * New hardware has been deployed for hg.mozilla.org and hgssh servers. * A new read-only unified Firefox repo has been created: https://hg.mozilla.org/mozilla-unified. See http://bit.ly/29En8Ra * Pushes to hg.mozilla.org now generate Pulse notifications: http://bit.ly/2a3Qw1w * The “autoland” repository has been created and is receiving pushes from landings initiated via MozReview. '''General Automation''' * Performance of xpcshell tests on Windows has been improved by avoiding scans by Windows Defender: https://mzl.la/29JBqlu * Reftest performance has been increased significantly (especially in automation) by disabling the (unneeded) Places db writes that were occurring during the tests. * A mozlint library has been created, which makes it easy to add linter jobs in-tree: https://mzl.la/29Epxv7 * ActiveData is now ingesting TaskCluster data. ActiveData is also being used to prototype an alternate view on intermittent test failures: https://mzl.la/29A4Yjb (link is slow to load) * PulseGuardian now shows queue bindings for your users. _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
