On 2014-09-05, 11:05 AM, Gregory Szorc wrote:
On 9/5/14 7:27 AM, Ehsan Akhgari wrote:
On 2014-09-04, 5:31 PM, Gregory Szorc wrote:
On 9/4/14 2:13 PM, Mike Hommey wrote:
On Thu, Sep 04, 2014 at 01:25:29PM -0700, Gregory Szorc wrote:
There are some things jotted down at
https://etherpad.mozilla.org/build-system-goals
But that hasn't been updated in months.
For immediate goals, I'd like to see the FINAL_TARGET situation
cleaned up
so moz.build can start managing things there (FINAL_TARGET is often
dist/bin). Teaching moz.build about things under dist/bin allows us
to do
less work at build time and that makes builds faster. I reckon we
could get
no-op builds under 15s on non-Windows by EOY if we targeted this area
for
improvement. Some work in bug 1058051 decreased no-op times by ~20% by
beating around the bush. I haven't had time to finish those
patches. It
sounds like glandium would like the proper XPI handling in place
instead of
the workarounds in those patches.
A good first milestone is doing XPI declaration and build rule
generation in
moz.build. See bug 988938. Everything related to XPI_NAME is a giant
hack.
See junk like netwerk/test/httpserver/Makefile.in. I was planning to
work on
this as soon as I can, but I likely won't have time until October.
I think the more immediate thing to do is to move things out of the
libs
tier into a new parallel tier with a better name.
Definitely. There are tons of install rules in libs:: today that can be
moved to a new parallel tier. That's great fodder for a non-expert.
Should result in immediate speed wins.
Can you please provide any pointers on what this is about more
specifically and what an example patch would look like?
https://hg.mozilla.org/mozilla-central/file/dddbe46f3ceb/python/mozbuild/mozbuild/backend/recursivemake.py#l486,
config/recurse.mk, and Makefile.in are what you'll want to update.
Commits touching this code:
https://hg.mozilla.org/mozilla-central/rev/0300f04572b6
https://hg.mozilla.org/mozilla-central/rev/0cd3fbbf21f8
It's kind of wonky and hard to follow. Essentially, you write root.mk
and root-deps.mk to contain the set of directories in a tier and the
dependencies for each directory in that tier. Tiers like export are
nearly fully parallel (most targets in root-deps.mk have no
dependencies). That's all done in recursivemake.py.
When you add a new tier, you'll need to hook up the plumbing in
config/recurse.mk and Makefile.in.
If you implement the directory "selection" like is done for libs, we'll
only traverse over directories that have a libs:: target. This means
there is practically no penalty for adding tiers.
Once the new tier is in place, you can start iterating through
Makefile.in and replace "libs::" with "newtier::" for rules that don't
have any order dependencies. You can also look for things in rules.mk
and recursivemake.py that do stuff in libs:: and start converting those
over. I imagine we could get no-op to under 30s easy. It's just a bit
labor intensive and error prone. We suspect there are a ton of traversal
order dependencies in the libs tier. Moving to parallel will tease these
out via "random" build failures. Tread carefully.
Thanks!
So, I can only see three places in rules.mk where we seem to do things
in libs::, and I cannot find anything else in recursivemake.py or
Makefile.in. Am I missing something? Or are those three rules the only
work to do in a new tier?
Also, I'm not really sure where to start from here. I think given the
above one can create a new tier, but I don't know what things would be
safe to move into it, etc...
_______________________________________________
dev-builds mailing list
dev-builds@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds
