On Wed, Aug 16, 2017 at 1:27 PM Gregory Szorc <g...@mozilla.com> wrote:
> FWIW, the general process Mercurial took for Python 3 compatibility looked
> like:
>
> 1) Make all .py files tokenize/ast compile on Python 3
> 2) Make all .py files import cleanly on Python 3
> 3) Make all .py files run properly on Python 3
>
There's been some progress on the step 1) front. There will soon be two
linters
running under the 'py-compat' task called py2 and py3 (mach lint -l py2 -l
py3).
The py3 linter makes sure all files it checks can be parsed with python3.
The py2
linter makes sure all files it checks can be parsed with python2 and
additionally
enforces all files have:
from __future__ import print_function, absolute_import
in them. As of now, almost all python files are being excluded from these
linters,
so please take a moment to enable these checks on the files you are
responsible
for. To do this, remove your directory from the exclude lists under:
tools/lint/py2.yml
tools/lint/py3.yml
This also makes great [good-first-bug] material.
I think that worked out pretty well and recommend repeating the pattern at
> Mozilla. Having a Python module import cleanly unblocks usage of just a
> single function in that module from Python 3, which for many consumers is
> all you need. In terms of tests, we'll want to establish "gates" where once
> a module passes a tokenize/ast or import threshold, it stays that way.
>
>
> > 2017-08-16 12:54 GMT-04:00 Gregory Szorc <g...@mozilla.com>:
> >
> >> In the past few weeks, a few independent projects have inquired about
> >> using Python 3 for Firefox build/development tooling.
> >>
> >> tl;dr we now have bug 1388447 (aliased as "buildpython3") tracking
> >> everything Python 3 related to Firefox development. Please use it.
> >>
> >> My thoughts on Python 3 and Firefox follow.
> >>
> >> Python 2 will stop being supported in 2020 (exact date TBD). In theory,
> >> there will be a 2.7.X release in 2020 and no more official Python 2
> >> releases will be made. It is therefore prudent for any
> actively-developed
> >> Python code to transition away from Python 2 so it remains on a
> supported
> >> programming language. Python 2 won't disappear over night and is likely
> to
> >> be a fixture of operating system installs for years to come. But as time
> >> goes by, it will be more and more annoying having to support legacy
> Python
> >> 2 code. Any developer who spends a lot of time writing Python will
> likely
> >> scoff at the thought of maintaining Python 2 code after they are
> accustomed
> >> to Python 3. Python 2 code will tend to rot from neglect. And, running
> >> Python 2 code will likely expose you to unpatched security
> vulnerabilities.
> >> It's best to be off of Python 2 by its end of life in 2020.
> >>
> >> That being said, you don't need to transition to Python 3 and nobody is
> >> forcing you to transition to Python 3. While there are gains from
> >> transitioning, it is a lot of work. In many cases, porting is of the
> same
> >> scope as a rewrite and therefore many of the reasons why rewrites are a
> bad
> >> idea apply to porting to Python 3. So it would be prudent to conduct a
> >> cost-benefit analysis on porting, paying special attention to what other
> >> work is being neglected because a port is being done.
> >>
> >> IMO Python 3 is a superior programming language to Python 2. The
> run-time
> >> performance of Python 3 is substantially better than Python 2 as of the
> 3.6
> >> release (although start-up time is still an issue). There are compelling
> >> Python 3 features that make it attractive. These include support for
> type
> >> hinting, which combined with a type aware editor like PyCharm (which
> >> Mozilla employees can get a license for), should facilitate easier
> hacking
> >> on large code bases like the Firefox build system. Asyncio allows much
> >> higher concurrency for I/O bound tasks (which there are several of in
> the
> >> build system and tools). And there are a ton of standard library
> >> improvements that reduce code complexity, attain perf wins, lower
> >> development overhead, etc. In aggregate, these make large, actively
> >> developed, performance sensitive code the most lucrative targets to
> port.
> >>
> >> We should be encouraging the use of Python 3 for Firefox tooling. I dare
> >> say that new Python code should strive to be Python 3 only if possible,
> as
> >> new Python 2 in 2017 is a form of technical debt. At the very least, I
> >> encourage all newly-written Python code to be as Python 3 compatible as
> >> possible to minimize future porting work.
> >>
> >> Requiring Python 3 to build Firefox incurs a non-trivial amount of work,
> >> especially to downstream packagers. Bug 636155 tracks *requiring*
> Python 3
> >> to build Firefox. A summary of my thoughts are captured in comment #9.
> That
> >> milestone likely won't happen until 2018Q2 or later.
> >>
> >> However, we don't need to require Python 3 to build Firefox in order to
> >> use Python 3 in Firefox tooling. Non-essential development activities
> can
> >> adopt Python 3 sooner. Examples of non-essential development activities
> >> include mach commands not used by the build system, such as `mach
> watch`.
> >> Strictly speaking, I even consider testing harnesses non-essential, as
> they
> >> aren't required to *build*. (Although I'm not sure how much testing
> >> downstream packagers do and we need to be cognizant about introducing a
> >> Python 3 requirement on them.)
> >>
> >> Bug 1385381 recently landed to detect Python 3 in the build system. The
> >> PYTHON3 subst variable, if defined, will point to a Python 3.5+
> executable.
> >> The MozbuildObject base class now exposes a `python3` attribute. It
> returns
> >> a 2-tuple of (path, version). So, from mach command classes inheriting
> from
> >> MachCommandBase and various build system contexts, you can now do
> something
> >> like:
> >>
> >> python3 = self.python3[0]
> >> if not python3:
> >> raise Exception('Python 3 not found')
> >> subprocess.check_call([python3, ...])
> >>
> >> Mach itself is still Python 2 and likely will be for some time. There
> >> have been some discussions around making it work with Python 3. We may
> even
> >> facilitate light magic to allow mach @Command functions to
> transparently be
> >> invoked as Python 3. But for now, you'll have to invoke a separate
> Python 3
> >> process from mach commands to use Python 3.
> >>
> >> There is a group interested in porting various Mozilla packages (like
> >> mozbase) so they are Python 3 compatible. Porting things like mozprocess
> >> will be a prerequisite for making large amounts of the Python in the
> >> Firefox repo Python 3 compatible.
> >>
> >> Managing a flag day transition to Python 3 across the entire Firefox
> code
> >> base is not feasible: it is just too much work. The conversion to
> Python 3
> >> will be gradual and take years. Expect Python 2 and 3 to run
> side-by-side
> >> for years, well past 2020. Components will adopt different schedules and
> >> priorities for the transition. Early adopters of Python 3 will likely be
> >> hindered by various package dependencies not yet working on Python 3.
> For
> >> this reason, early Python 3 efforts should be focused on shared packages
> >> like mozbase and new code that doesn't need too much of the in-tree
> Python
> >> (which is likely unported).
> >>
> >> Because Python 2 and 3 will run side-by-side for years, shared packages
> >> should strive to run on both Python 2 and 3. This is achievable in many
> >> cases. Packages like six do a great job of wallpapering over
> differences.
> >> In cases where the same code can't run on both 2 and 3 simultaneously,
> we
> >> can look at using source rewriting. I have some experience with this and
> >> you can reach out if you have any questions.
> >>
> >> Again, nobody is forcing you to use or port to Python 3. But Python 3 is
> >> the future and anyone writing or maintaining a significant amount of
> Python
> >> should be cognizant of the technical debt being incurred by continuing
> to
> >> write Python 2 in 2017 and beyond. We should start thinking "Python 3
> >> first" and start planning for a world that is "Python 3 everywhere." For
> >> most of you, it should be business as usual for the next several months.
> >> But once more Python 3 "infrastructure" is in place in Firefox tooling,
> >> expect the calls for Python 3 to grow louder. There's no stopping the
> >> Python 3 train. So please be aware that it's coming.
> >>
> >> _______________________________________________
> >> dev-builds mailing list
> >> dev-builds@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-builds
> >>
> >>
> >
> _______________________________________________
> tools mailing list
> to...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/tools
>
_______________________________________________
dev-builds mailing list
dev-builds@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds
