> On Aug 29, 2017, at 1:22 PM, Olmstead, Don <[email protected]> wrote: > > Sure but in a corporate environment it can be a really big win. All of our > developers have their machines hooked up to the distributed build system and > we also have servers hanging out that can be used in the build. This sort of > thing doesn't require anyone to change how they write their code which is > something a unity build ends up requiring. > > In terms of WebKit development the only places it falls apart are in the > source code generation and the link. > > Really I was just curious if anything like that was discussed because all of > this requires pretty big changes to source code. Additionally there's other > fun things that can happen around includes when doing a unity build where a > file will stop being able to be built in isolation. The same thing has > happened with WebCorePrefix where a number of files can't be built without it. Maybe it’d be worth having an EWS bot builds the non-unity build so we don’t get into the same situation here.
- Saam > > -----Original Message----- > From: Konstantin Tokarev [mailto:[email protected]] > Sent: Tuesday, August 29, 2017 11:22 AM > To: Olmstead, Don <[email protected]>; Keith Miller > <[email protected]>; WebKit Development Mailing List > <[email protected]> > Subject: Re: [webkit-dev] Growing tired of long build times? Check out this > awesome new way to speed up your build... soon (HINT: It's not buying a new > computer) > > > > 29.08.2017, 21:19, "Olmstead, Don" <[email protected]>: >> Did you happen to look at any sort of distributed build system? We use >> distributed builds here for WebKit and a full rebuild is pretty reasonable. >> That would negate the need to change how everything is done to get a unity >> build going. > > This isn't helpful for contributors who have only one machine at their > disposal > >> >> From: webkit-dev >> [mailto:[email protected]] On >> Behalf Of Keith Miller >> Sent: Monday, August 28, 2017 5:37 PM >> To: WebKit Development Mailing List <[email protected]> >> Subject: [webkit-dev] Growing tired of long build times? Check out this >> awesome new way to speed up your build... soon (HINT: It's not buying a new >> computer) >> >> Hello fellow Webkittens, >> >> Are you growing tired of long cat naps while waiting 45 minutes for WebKit >> to build? (I’ve definitely never done this 🤐!) >> >> Do you want WebKit to build up to 3-4x faster on a clean build?* >> >> Does seeing your incremental build… stay the same sound good? >> >> You’ll be purring with excitement after you switch to unified source >> builds!** >> >> * Building JSC with CMake, including CMake configuration time, went from >> ~8m15s to ~2m30s on my MBP using unified sources only on cpp files in >> JavaScriptCore. Final results on WebKit may vary. Using unified sources may >> cause sudden feelings of euphoria and productivity (or grumpiness from lack >> of naps…). Not responsible for any bugs in extra patches produced by using >> unified source builds (excluding build system related bugs) except where >> required by law ([email protected]). >> >> ** Soon. Unified source builds haven’t been landed yet. >> >> How are things going to change? Great question! Let’s find out! >> >> When you fire off the build script it will automagically paw your source >> files into bundles. The size of each bundle is speculatively going to be 8 >> .cpp files but that’s up for experimentation. From my testing so far, the >> compile time for a bundle is at most 20% (~6s vs ~7s for the Air directory) >> longer than the longest file it includes. Given that most of the build time >> in incremental builds is scanning dependencies, this change is probably only >> barely noticeable. >> >> Bundling happens as part of the CMake configuration process, in the CMake >> build. In the Xcode build it’s more complicated since we cannot dynamically >> choose the files we are going to compile. The only solution I know of is to >> pre-list a bunch of files for the build to dump files into. Occasionally, >> we’ll need to add more files to the build list. >> >> When you add or remove a file in the project, the cost is higher. The >> current plan to bundle source files is by directory. This means that when a >> .cpp file is added or removed you will rebalance the bundles in its >> directory, and you will have to rebuild up to the full directory that file >> is in. >> >> My goal is to make all of this invisible to developers as much as possible. >> I believe, for the most part, things should operate mostly as they have >> before. As the details on unified source builds get finalized, I’m sending >> out a separate email documenting any workflow changes. I’ll also do my best >> to answer any questions or concerns. >> >> “But Keith, am I going to have to make major changes to the way I develop >> WebKit?” Yes and, hopefully, no. There is one major common workflow change >> that comes with unified sources, the naming of static variables and >> functions. Since C++ does not have a way to only declare a static function >> or variable in some restricted top level scope we are going to need to >> create some work around. The choice I’m thinking we’ll go with is to have an >> auto-generated macro value FILENAME, which is similar to __FILE__ except >> that it doesn’t have file extension (We can’t use FILE since that’s actually >> a type in C 🙁). This macro would be redefined in the unified source directly >> above the file that is about to be included. Then in each cpp file that has >> static functions/variables developers would have: >> >> namespace FILENAME { >> >> static int myVariable { 0 }; >> >> static int myFunction() { return myVariable; } >> >> } >> >> The advantage of using a macro over textually writing the filename is that >> it makes code more portable between files. Also, once people get used to the >> concept of FILENAME in the code base it will, hopefully become obvious what >> it means. The main downside to this is using FILENAME in a header would give >> you whatever cpp file first included you (if only we got the compiler magic >> of __FILE__...). We would probably need to prohibit using FILENAME in .h >> files. Unfortunately, you cannot "using FILENAME;” because it will export >> all the values in each subsequent namespace. e.g. >> >> namespace JSC { >> >> namespace Foo { >> >> static int myVar { 0 }; >> >> } >> >> using Foo; >> >> } >> >> namespace JSC { >> >> namespace Bar { >> >> static int myVar { 0 }; >> >> } >> >> using Bar; >> >> } >> >> does not compile. >> >> Does anyone have other name recommendations for the macro name other than >> FILENAME? I went with file name since it describes what the substituted >> value will be the best, which is helpful for debugging. The other options I >> considered were FILE_LOCAL and PER_FILE. >> >> Are there other issues that people think might occur with unified sources? >> Note, It’s also likely that there are some source files that need to be >> excluded from the unified source builds for various reasons. >> >> “Is there anything else I need to do with unified sources?”, Yes! Enjoy the >> faster builds! >> >> To help visualize how much of a difference unified sources can make for you, >> I’ve illustrated this helpful graph on the WebKit developer experience: >> >> * Due to size constraints this graph is not to scale. >> >> Cheers, >> >> Keith >> >> P.S. >> >> For those interested there was a long process in deciding how to improve >> builds: >> >> The first thing I looked at was using C++ modules. >> >> There were a couple of problems with C++ modules. First, different compliers >> support modules in different ways. This was not a major concern since we >> probably would have gone with Clang’s version of modules. As far as I know, >> almost all developers either use clang or could likely switch to clang for >> development. The bigger problem with Clang modules is that it seemed like we >> would have a rough early adopter problem. Additionally, since a lot of >> Apple’s internal SDK’s don’t seem to support modules it was likely to be a >> major slog to get everything working correctly. In a few years I think it >> will probably be worth looking at C++ modules again as, in theory, C++ >> modules should be better than unified sources. >> >> Since we decided against C++ modules that really only leaves us with some >> form of unified sources. Obviously, doing all in one builds would be out of >> the question. While all in ones would have been great for clean builds, it >> also means incremental builds are essentially the same as clean builds. So, >> I concluded that we should go with some kind of automated unified build file >> solution. An automated unified build file solution means that most of the >> time the build system is opaque to developers (well as much as is possible >> with a unified source build). >> >> Since the static variable naming problem was likely to be the biggest issue >> with unified sources, it was important to create a system that made this a >> pain free as possible. There were other options we could go with. We could >> have a script that parses .cpp files and looks for static variables with the >> same name and yells at you. This is pretty annoying and wouldn’t help actual >> development when people hit an error. For example, I know I never run the >> check-webkit-style script until I’m putting a patch up for review. >> , >> >> _______________________________________________ >> webkit-dev mailing list >> [email protected] >> https://lists.webkit.org/mailman/listinfo/webkit-dev > > > -- > Regards, > Konstantin > _______________________________________________ > webkit-dev mailing list > [email protected] > https://lists.webkit.org/mailman/listinfo/webkit-dev _______________________________________________ webkit-dev mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-dev
