Hi, We are attempting to use GNU Make 4.2.1 to build a large software tree (~290000 targets).
The build hosts we are testing on are (virtualized) quad core X86-64 machines running RHEL 6. Our code tree is currently on "locally perceived" storage. We compiled GNU Make 4.2.1 from sources. Our main concern at this moment is the time needed for "nil incrementals", i.e. the time needed to establish that nothing has changed, so nothing needs to be done. If we put all dependencies in the Makefile itself or in one single included makefile, we manage to do a nil incremental in about 12 seconds. But we plan to use auto-generated dependencies and thus include a .d file in our main Makefile for each target. Currently auto dependency generation is possible for about 90000 of our targets. Nil incremental builds cost increases to about 140 seconds in that case, which is quite unacceptable for our usage. We found that this overhead increases approximately quadratically with the number of included dependency files. It appears that this behavior is caused by the need to maintain the variable MAKEFILE_LIST. The names of all included makefiles are appended one by one to the ever growing string representing this variable.This also explains why memcpy() and strlen() champion the profiled runs we did and also the quadratic behavior we observed. When we change the source (read.c) so that the MAKEFILE_LIST is not maintained at all, the overhead is gone and we are back to about 12 seconds again. Are there any ideas (or opinions) about how to get rid of this type of undesirable overhead? One of our own ideas is to provide a special variant of -include (called e.g. dinclude?), intended to include just .d files, without maintaining MAKEFILE_LIST. On the other hand it might be feasible to improve the efficiency of appending to a variable. Any suggestions? By the way, Ninja needs about 7 seconds for a nil incremental with about 90000 .d files. Kind regards, Hans Ottevanger _______________________________________________ Bug-make mailing list [email protected] https://lists.gnu.org/mailman/listinfo/bug-make
