On 2016-11-21 1:08 PM, Gregory Szorc wrote: > On Sat, Nov 19, 2016 at 1:32 PM, Ehsan Akhgari <ehsan.akhg...@gmail.com > <mailto:ehsan.akhg...@gmail.com>> wrote: > > On 2016-11-19 4:26 PM, Mike Hommey wrote: > > On Sat, Nov 19, 2016 at 04:12:22PM -0500, Ehsan Akhgari wrote: > >> On 2016-11-18 7:22 PM, Gregory Szorc wrote: > >>> On Fri, Nov 18, 2016 at 3:50 PM, Ehsan Akhgari > <ehsan.akhg...@gmail.com <mailto:ehsan.akhg...@gmail.com> > >>> <mailto:ehsan.akhg...@gmail.com > <mailto:ehsan.akhg...@gmail.com>>> wrote: > >>> > >>> Here are the moz.build processing times on my machine when I > add one > >>> line to a LOCAL_INCLUDES in one moz.build file: > >>> > >>> 0:02.97 Reticulating splines... > >>> 0:37.16 Finished reading 3131 moz.build files in 23.09s > >>> 0:37.16 Processed into 9325 build config descriptors in 3.62s > >>> 0:37.16 RecursiveMake backend executed in 6.50s > >>> 0:37.16 2951 total backend files; 0 created; 2 updated; 2949 > >>> unchanged; 0 deleted; 54 -> 1110 Makefile > >>> 0:37.16 FasterMake backend executed in 0.50s > >>> 0:37.16 12 total backend files; 0 created; 0 updated; 12 > unchanged; 0 > >>> deleted > >>> 0:37.16 Total wall time: 34.22s; CPU time: 16.20s; > Efficiency: 47%; > >>> Untracked: 0.50s > >>> > >>> > >>> 0:37.89 Finished reading 3131 moz.build files in 21.90s > >>> 0:37.89 Processed into 9327 build config descriptors in 3.82s > >>> 0:37.89 RecursiveMake backend executed in 7.85s > >>> 0:37.89 2951 total backend files; 0 created; 4 updated; 2947 > >>> unchanged; 0 deleted; 54 -> 1110 Makefile > >>> 0:37.89 FasterMake backend executed in 0.59s > >>> 0:37.89 12 total backend files; 0 created; 0 updated; 12 > unchanged; 0 > >>> deleted > >>> 0:37.89 Total wall time: 34.70s; CPU time: 17.70s; > Efficiency: 51%; > >>> Untracked: 0.55s > >>> > >>> The clear offender is "reading moz.build files". What's > involved in > >>> this process? Is there some low hanging fruit for making it > faster? > >>> Are there things non-build developers can do (such as > reducing the > >>> number of moz.build files) that would help? > > > > Please note that "reading moz.build files" is actually misleading, > > because usually half of the time is spent not reading moz.build files. > > That also seems like a lot of time to spend on reading a "small" > number > > of files (because the printed number of files is actually wrong, > see bug > > 1208380). It would be interesting to see a profile for that "reading > > moz.build files" phase. > > If there is an easy thing I can do to capture such a profile, please let > me know. I don't know where in the code you're referring to, and I have > also never really profiled Python code. :-) > > > You can profile any Python process by running it with: > > $ python -mcProfile <program> <args> > > e.g. > > $ python -mcProfile mach build-backend > > You probably want to sort the output using "tottime" or "cumtime" (total > and cumulative) respectively. e.g. > > $ python -mcProfile -s cumtime mach build-backend
Here is a profile: <https://gist.github.com/ehsan/b593c20456886be03c0109e92b4fbf4f> (For some reason now it takes ~15s to read the moz.build files on the same machine...) _______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
