In the past few weeks, a few independent projects have inquired about using
Python 3 for Firefox build/development tooling.
tl;dr we now have bug 1388447 (aliased as "buildpython3") tracking
everything Python 3 related to Firefox development. Please use it.
My thoughts on Python 3 and Firefox follow.
Python 2 will stop being supported in 2020 (exact date TBD). In theory,
there will be a 2.7.X release in 2020 and no more official Python 2
releases will be made. It is therefore prudent for any actively-developed
Python code to transition away from Python 2 so it remains on a supported
programming language. Python 2 won't disappear over night and is likely to
be a fixture of operating system installs for years to come. But as time
goes by, it will be more and more annoying having to support legacy Python
2 code. Any developer who spends a lot of time writing Python will likely
scoff at the thought of maintaining Python 2 code after they are accustomed
to Python 3. Python 2 code will tend to rot from neglect. And, running
Python 2 code will likely expose you to unpatched security vulnerabilities.
It's best to be off of Python 2 by its end of life in 2020.
That being said, you don't need to transition to Python 3 and nobody is
forcing you to transition to Python 3. While there are gains from
transitioning, it is a lot of work. In many cases, porting is of the same
scope as a rewrite and therefore many of the reasons why rewrites are a bad
idea apply to porting to Python 3. So it would be prudent to conduct a
cost-benefit analysis on porting, paying special attention to what other
work is being neglected because a port is being done.
IMO Python 3 is a superior programming language to Python 2. The run-time
performance of Python 3 is substantially better than Python 2 as of the 3.6
release (although start-up time is still an issue). There are compelling
Python 3 features that make it attractive. These include support for type
hinting, which combined with a type aware editor like PyCharm (which
Mozilla employees can get a license for), should facilitate easier hacking
on large code bases like the Firefox build system. Asyncio allows much
higher concurrency for I/O bound tasks (which there are several of in the
build system and tools). And there are a ton of standard library
improvements that reduce code complexity, attain perf wins, lower
development overhead, etc. In aggregate, these make large, actively
developed, performance sensitive code the most lucrative targets to port.
We should be encouraging the use of Python 3 for Firefox tooling. I dare
say that new Python code should strive to be Python 3 only if possible, as
new Python 2 in 2017 is a form of technical debt. At the very least, I
encourage all newly-written Python code to be as Python 3 compatible as
possible to minimize future porting work.
Requiring Python 3 to build Firefox incurs a non-trivial amount of work,
especially to downstream packagers. Bug 636155 tracks *requiring* Python 3
to build Firefox. A summary of my thoughts are captured in comment #9. That
milestone likely won't happen until 2018Q2 or later.
However, we don't need to require Python 3 to build Firefox in order to use
Python 3 in Firefox tooling. Non-essential development activities can adopt
Python 3 sooner. Examples of non-essential development activities include
mach commands not used by the build system, such as `mach watch`. Strictly
speaking, I even consider testing harnesses non-essential, as they aren't
required to *build*. (Although I'm not sure how much testing downstream
packagers do and we need to be cognizant about introducing a Python 3
requirement on them.)
Bug 1385381 recently landed to detect Python 3 in the build system. The
PYTHON3 subst variable, if defined, will point to a Python 3.5+ executable.
The MozbuildObject base class now exposes a `python3` attribute. It returns
a 2-tuple of (path, version). So, from mach command classes inheriting from
MachCommandBase and various build system contexts, you can now do something
like:
python3 = self.python3[0]
if not python3:
raise Exception('Python 3 not found')
subprocess.check_call([python3, ...])
Mach itself is still Python 2 and likely will be for some time. There have
been some discussions around making it work with Python 3. We may even
facilitate light magic to allow mach @Command functions to transparently be
invoked as Python 3. But for now, you'll have to invoke a separate Python 3
process from mach commands to use Python 3.
There is a group interested in porting various Mozilla packages (like
mozbase) so they are Python 3 compatible. Porting things like mozprocess
will be a prerequisite for making large amounts of the Python in the
Firefox repo Python 3 compatible.
Managing a flag day transition to Python 3 across the entire Firefox code
base is not feasible: it is just too much work. The conversion to Python 3
will be gradual and take years. Expect Python 2 and 3 to run side-by-side
for years, well past 2020. Components will adopt different schedules and
priorities for the transition. Early adopters of Python 3 will likely be
hindered by various package dependencies not yet working on Python 3. For
this reason, early Python 3 efforts should be focused on shared packages
like mozbase and new code that doesn't need too much of the in-tree Python
(which is likely unported).
Because Python 2 and 3 will run side-by-side for years, shared packages
should strive to run on both Python 2 and 3. This is achievable in many
cases. Packages like six do a great job of wallpapering over differences.
In cases where the same code can't run on both 2 and 3 simultaneously, we
can look at using source rewriting. I have some experience with this and
you can reach out if you have any questions.
Again, nobody is forcing you to use or port to Python 3. But Python 3 is
the future and anyone writing or maintaining a significant amount of Python
should be cognizant of the technical debt being incurred by continuing to
write Python 2 in 2017 and beyond. We should start thinking "Python 3
first" and start planning for a world that is "Python 3 everywhere." For
most of you, it should be business as usual for the next several months.
But once more Python 3 "infrastructure" is in place in Firefox tooling,
expect the calls for Python 3 to grow louder. There's no stopping the
Python 3 train. So please be aware that it's coming.
_______________________________________________
dev-builds mailing list
dev-builds@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds
