On 2016-11-22 7:39 PM, Gregory Szorc wrote: > On Mon, Nov 21, 2016 at 2:06 PM, Ehsan Akhgari <ehsan.akhg...@gmail.com > <mailto:ehsan.akhg...@gmail.com>> wrote: > > On 2016-11-21 1:08 PM, Gregory Szorc wrote: > > On Sat, Nov 19, 2016 at 1:32 PM, Ehsan Akhgari <ehsan.akhg...@gmail.com > <mailto:ehsan.akhg...@gmail.com> > > <mailto:ehsan.akhg...@gmail.com <mailto:ehsan.akhg...@gmail.com>>> > wrote: > > > > On 2016-11-19 4:26 PM, Mike Hommey wrote: > > > On Sat, Nov 19, 2016 at 04:12:22PM -0500, Ehsan Akhgari wrote: > > >> On 2016-11-18 7:22 PM, Gregory Szorc wrote: > > >>> On Fri, Nov 18, 2016 at 3:50 PM, Ehsan Akhgari > > <ehsan.akhg...@gmail.com <mailto:ehsan.akhg...@gmail.com> > <mailto:ehsan.akhg...@gmail.com <mailto:ehsan.akhg...@gmail.com>> > > >>> <mailto:ehsan.akhg...@gmail.com > <mailto:ehsan.akhg...@gmail.com> > > <mailto:ehsan.akhg...@gmail.com > <mailto:ehsan.akhg...@gmail.com>>>> wrote: > > >>> > > >>> Here are the moz.build processing times on my machine > when I > > add one > > >>> line to a LOCAL_INCLUDES in one moz.build file: > > >>> > > >>> 0:02.97 Reticulating splines... > > >>> 0:37.16 Finished reading 3131 moz.build files in 23.09s > > >>> 0:37.16 Processed into 9325 build config descriptors > in 3.62s > > >>> 0:37.16 RecursiveMake backend executed in 6.50s > > >>> 0:37.16 2951 total backend files; 0 created; 2 > updated; 2949 > > >>> unchanged; 0 deleted; 54 -> 1110 Makefile > > >>> 0:37.16 FasterMake backend executed in 0.50s > > >>> 0:37.16 12 total backend files; 0 created; 0 > updated; 12 > > unchanged; 0 > > >>> deleted > > >>> 0:37.16 Total wall time: 34.22s; CPU time: 16.20s; > > Efficiency: 47%; > > >>> Untracked: 0.50s > > >>> > > >>> > > >>> 0:37.89 Finished reading 3131 moz.build files in 21.90s > > >>> 0:37.89 Processed into 9327 build config descriptors > in 3.82s > > >>> 0:37.89 RecursiveMake backend executed in 7.85s > > >>> 0:37.89 2951 total backend files; 0 created; 4 > updated; 2947 > > >>> unchanged; 0 deleted; 54 -> 1110 Makefile > > >>> 0:37.89 FasterMake backend executed in 0.59s > > >>> 0:37.89 12 total backend files; 0 created; 0 > updated; 12 > > unchanged; 0 > > >>> deleted > > >>> 0:37.89 Total wall time: 34.70s; CPU time: 17.70s; > > Efficiency: 51%; > > >>> Untracked: 0.55s > > >>> > > >>> The clear offender is "reading moz.build files". What's > > involved in > > >>> this process? Is there some low hanging fruit for > making it > > faster? > > >>> Are there things non-build developers can do (such as > > reducing the > > >>> number of moz.build files) that would help? > > > > > > Please note that "reading moz.build files" is actually > misleading, > > > because usually half of the time is spent not reading > moz.build files. > > > That also seems like a lot of time to spend on reading a "small" > > number > > > of files (because the printed number of files is actually wrong, > > see bug > > > 1208380). It would be interesting to see a profile for that > "reading > > > moz.build files" phase. > > > > If there is an easy thing I can do to capture such a profile, > please let > > me know. I don't know where in the code you're referring to, > and I have > > also never really profiled Python code. :-) > > > > > > You can profile any Python process by running it with: > > > > $ python -mcProfile <program> <args> > > > > e.g. > > > > $ python -mcProfile mach build-backend > > > > You probably want to sort the output using "tottime" or "cumtime" > (total > > and cumulative) respectively. e.g. > > > > $ python -mcProfile -s cumtime mach build-backend > > Here is a profile: > <https://gist.github.com/ehsan/b593c20456886be03c0109e92b4fbf4f > <https://gist.github.com/ehsan/b593c20456886be03c0109e92b4fbf4f>> > > (For some reason now it takes ~15s to read the moz.build files on the > same machine... > > > Doh - I forgot `mach build-backend` invokes a separate Python process - > one that doesn't have a profiler attached. You'll need to run: > > $ cd objdir > $ mach python -mcProfile -s cumtime config.status
Here is mine: <https://gist.github.com/ehsan/ad4e56f2d2f70e3e6c578a2517afbc15> _______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
