On Sat, Nov 19, 2016 at 04:12:22PM -0500, Ehsan Akhgari wrote: > On 2016-11-18 7:22 PM, Gregory Szorc wrote: > > On Fri, Nov 18, 2016 at 3:50 PM, Ehsan Akhgari <ehsan.akhg...@gmail.com > > <mailto:ehsan.akhg...@gmail.com>> wrote: > > > > Here are the moz.build processing times on my machine when I add one > > line to a LOCAL_INCLUDES in one moz.build file: > > > > 0:02.97 Reticulating splines... > > 0:37.16 Finished reading 3131 moz.build files in 23.09s > > 0:37.16 Processed into 9325 build config descriptors in 3.62s > > 0:37.16 RecursiveMake backend executed in 6.50s > > 0:37.16 2951 total backend files; 0 created; 2 updated; 2949 > > unchanged; 0 deleted; 54 -> 1110 Makefile > > 0:37.16 FasterMake backend executed in 0.50s > > 0:37.16 12 total backend files; 0 created; 0 updated; 12 unchanged; 0 > > deleted > > 0:37.16 Total wall time: 34.22s; CPU time: 16.20s; Efficiency: 47%; > > Untracked: 0.50s > > > > > > 0:37.89 Finished reading 3131 moz.build files in 21.90s > > 0:37.89 Processed into 9327 build config descriptors in 3.82s > > 0:37.89 RecursiveMake backend executed in 7.85s > > 0:37.89 2951 total backend files; 0 created; 4 updated; 2947 > > unchanged; 0 deleted; 54 -> 1110 Makefile > > 0:37.89 FasterMake backend executed in 0.59s > > 0:37.89 12 total backend files; 0 created; 0 updated; 12 unchanged; 0 > > deleted > > 0:37.89 Total wall time: 34.70s; CPU time: 17.70s; Efficiency: 51%; > > Untracked: 0.55s > > > > The clear offender is "reading moz.build files". What's involved in > > this process? Is there some low hanging fruit for making it faster? > > Are there things non-build developers can do (such as reducing the > > number of moz.build files) that would help?
Please note that "reading moz.build files" is actually misleading, because usually half of the time is spent not reading moz.build files. That also seems like a lot of time to spend on reading a "small" number of files (because the printed number of files is actually wrong, see bug 1208380). It would be interesting to see a profile for that "reading moz.build files" phase. > > > > Is there any way for us to avoid reprocessing the world for every single > > moz.build file change? > > > > > > Please tell us a bit more about your machine. Because on my machine... > > > > Linux > > $ ./mach build-backend > > 0:00.08 > > /home/gps/src/firefox/obj-x86_64-pc-linux-gnu/_virtualenv/bin/python > > /home/gps/src/firefox/obj-x86_64-pc-linux-gnu/config.status > > Reticulating splines... > > Finished reading 3027 moz.build files in 3.79s > > Processed into 9028 build config descriptors in 1.54s > > RecursiveMake backend executed in 3.00s > > 2829 total backend files; 0 created; 0 updated; 2829 unchanged; 0 > > deleted; 54 -> 1065 Makefile > > FasterMake backend executed in 0.22s > > 12 total backend files; 0 created; 0 updated; 12 unchanged; 0 deleted > > Total wall time: 8.76s; CPU time: 7.26s; Efficiency: 83%; Untracked: 0.21s > > > > Windows 10 > > $ ./mach build-backend > > 0:00.55 > > c:/dev/src/firefox/obj-i686-pc-mingw32\_virtualenv\Scripts\python.exe > > c:/dev/src/firefox/obj-i686-pc-mingw32\config.status > > Reticulating splines... > > Finished reading 3041 moz.build files in 7.01s > > Processed into 9841 build config descriptors in 3.19s > > RecursiveMake backend executed in 3.81s > > 2860 total backend files; 0 created; 1 updated; 2859 unchanged; 0 > > deleted; 62 -> 1069 Makefile > > FasterMake backend executed in 0.38s > > 11 total backend files; 0 created; 1 updated; 10 unchanged; 0 deleted > > VisualStudio backend executed in 1.71s > > Generated Visual Studio solution at > > c:/dev/src/firefox/obj-i686-pc-mingw32\msvc\mozilla.sln > > Total wall time: 16.43s; CPU time: 16.43s; Efficiency: 100%; Untracked: > > 0.33s > > > > The numbers seem to indicate your machine is a bit old or slow? > > Hmm, my machine is a MacBook Pro (this is my main development machine). > It's a MacBook Pro (Retina, 15-inch, Mid 2014), 2.8Ghz processor, 16GB > Memory, 1TB APPLE SSD SM1024F, Running Sierra 10.12.1. > > My machine wasn't entirely idle, I had things like a Terminal with a > bunch of vim's open, Nightly, Chrome, Thunderbird, LimeChat. But these > numbers are in the usual ballpark I get. > > > FWIW, bug 1308982 is on our radar for speeding up moz.build reading. > > That's great! I think that will help people like me a lot. Thankfully > Chris seems to be currently working on it. > > But I'm still not sure why we can't only re-process the moz.build file > that has changed... Not sure there's a bug on file, but we've been talking about doing this for a long time. It's not exactly trivial, but we'll get there eventually. Mike _______________________________________________ dev-builds mailing list dev-builds@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-builds
