On 2018/01/24 9:50, Gregory Szorc wrote:
On Tue, Jan 23, 2018 at 4:34 PM, Gregory Szorc <g...@mozilla.com
<mailto:g...@mozilla.com>> wrote:
Speaking as a maintainer of the Firefox build system, we try to be
conservative about the set of dependencies required to build Firefox
from source. We recognize that every required dependency is a
potential source of failure and complexity. Dependencies can create
complications for packagers. From a build system perspective, it's
in our best interest to minimize the dependencies required to build
Firefox.
Our desire to keep things simple can be at odds with the wishes of
Firefox developers who wish to leverage new and exciting tools and
technologies. In short, the policy of minimizing build dependencies
can externalize costs onto overall Firefox development by hindering
people from using better tools. This adversely affects the
development velocity of Firefox and can cause product quality to suffer.
For a few years now, various pockets of Firefox development have
wanted to use Node.js as part of the development workflow. I
wouldn't say they directly want to use Node.js: they want to use the
large ecosystem of tools built around Node.js. But that's splitting
hairs. Our build system policy has been that leveraging Node.js and
its ecosystem for supplemental workflows is fine, but shipping a
build dependency on Node.js is not. Many Firefox developers are now
using Node.js in their day-to-day workflow for things like running
ESLint. We're using Node.js in CI. But we're not forcing people to
have Node.js installed to build Firefox.
Various groups have routed around the limitation that Node.js can't
be required to build Firefox. There are now Firefox features that do
require Node.js to build. However, the output from the Node.js tools
is checked into the Firefox source repository. So from the
perspective of the Firefox build system, Node.js doesn't exist and
therefore isn't a build dependency.
The status quo is not ideal. The more people I speak with, the more
apparent it is that our current policy of not allowing Node.js
tooling in the build system is causing more problems than it is
preventing. Speaking as the build system module owner and someone
who cares about developer workflows, tooling, and developer
productivity, I don't think the current policy is good for Mozilla.
I'd like to start a discussion about requiring Node.js to build Firefox.
What do I mean by "require Node.js?" Let's assume I mean having a
usable Node.js executable on the host system to be used during a
Firefox build.
What about npm or a package manager? I would strongly prefer to
limit the required dependency to Node.js itself. While the Firefox
build system would depend on 3rd party packages and tools (such as
Babel), I'm pretty insistent (as a build system maintainer) that
these dependencies be vendored into the Firefox source repository so
as to not incur a run-time dependency on a packaging service. I've
seen the chaos that "left-pad" caused. I don't fully trust the
security model of JavaScript package distribution. I don't think we
can risk the ability to build Firefox or the integrity of the
Firefox product by the availability and integrity of a 3rd party
packaging service. That may sound like a harsh thing to say. But
it's the posture we've applied elsewhere (such as to Python packages
and PyPI). So, this means that all JavaScript executed by Node.js as
part of the build would either be provided by Node.js itself or the
Firefox source repository. If we needed to use a package manager as
part of the build, that package manager could be vendored in the
Firefox repository along with other JavaScript libraries (not unlike
how we currently vendor Python's pip package manager).
A few people at Mozilla have poked at this problem already. We have
a general sense of where some pain points for us will be. We know
that getting modern versions of Node.js installed on various
distributions requires using 3rd party package repositories. We know
that Windows support could be painful. We know that installing
common packages can result of dozens if not hundreds of dependencies
being added. We know this could lead to us having to install
thousands of files as part of the Firefox build - an overhead I'm
not keen on seeing. We know all of this can add up to a significant
amount of overhead to support. (Yet it still feels like a lesser
problem than having people work around not being able to use Node.js
directly.)
What we don't generally know is the impact requiring Node.js would
have on downstream packagers. Our adoption of Rust last year was a
long and sometimes painful process. I have a feeling that requiring
Node.js would be a similar experience. But like Rust, I feel that
adopting Node.js is in the best long-term interest for Firefox
development velocity and product quality. I'm reluctant to cause
more hardship by introducing a new build dependency. But it's very
difficult to keep saying we can't use Node.js in the Firefox build
system. I wish I could say "we'll build SpiderMonkey and use that
instead." Unfortunately, many Node.js tools don't work with
SpiderMonkey, so that's not an option. Plus there are difficulties
with cross-compilation. As sad as it makes me to say it,
SpiderMonkey is not an option: Node.js is the only viable option.
If we require Node.js to build Firefox, what are the requirements,
desires, and hardships of downstream packagers and consumers of the
Firefox build system? Keep in mind that mozilla-central right now is
Firefox 60. That will become ESR 60 in May.
Quick follow-up. First to add mozilla-linux-taskforce. Second to note
that we almost certainly wouldn't make a change before Firefox 61. That
would give everyone only caring about ESR an extra ~1 year to deal with
fallout.
The following issue related to npm package affects only a subset of
mozilla developers: only the patch contributors who develop on their PCs
that run Debian GNU/Linux.
|mach bootstrap| to set up the development tools and niceties fails now
under Debian GNU/Linux.
This is because "npm" package is no longer in the standard package
archive :-(
This is happening since mid-December, I think.
I was trying to fix a problem caused by "setupterm" error: see
https://groups.google.com/forum/#!topic/mozilla.dev.apps.thunderbird/mzEu2KI9foA
I thought of upgrading the packages from the linux distribution and
then tried to see if refreshing tools by "mach bootstrap" would help
when I noticed the problem seeing |mach| execution failed.
Moral of the story is the fewer dependencies, the better.
However, as I noted, this particular issue affects only the minority
users of Debian GNU/Linux on their individual machines, I suppose.
(I hate to think mozilla forces its developers to use a different
distribution. I don't believe it does.)
Also, NPM developers have prepared a web page to cope with this:
https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
(Oops, according to the web page, Ubunto may be also affected by this.
If so, the future ubuntu users need to be warned. I am afraid that there
are many more ubuntu users than Debian users among patch developers.)
I think I will file both issues bugzilla: the latter for making it easy
for patch developers who use Debian GNU/Linux to become aware of the
solution. The former issue needs careful analysis ONCE I find the
solution to this particular issue on one of my PCs.
That said, "mach bootstrap" failing due to the missing npm package *IS A
NUISANCE*. I think a casual patch developer will quit doing so then and
there.
I am not a casual patch developer (doing this for the last several
years) and have used Debian distribution for more than 15 years.
AND I have been doing the software development on many hardware/software
including OS development itself close to 40 years now, and YET, I cannot
solve the original problem of setupterm failing right now :-(
(Admittedly, I am trying to solve the issue in my spare time which is
dwindling and so can't say exactly how hard the issue is. But usually,
this type of the problem can be fixed in a few days...)
So here it goes again:
Moral of the story is the fewer dependencies, the better.
If you expect non-paid free volunteer types to contribute to mozilla
software work, make the hurdle for development as low as possible.
TIA
_______________________________________________
dev-builds mailing list
dev-builds@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds
