On Tue, 2018-02-06 at 15:12 +0100, Hans Ottevanger wrote: > It appears that this behavior is caused by the need to maintain the > variable MAKEFILE_LIST. The names of all included makefiles are > appended one by one to the ever growing string representing this > variable.This also explains why memcpy() and strlen() champion the > profiled runs we did and also the quadratic behavior we observed. > When we change the source (read.c) so that the MAKEFILE_LIST is not > maintained at all, the overhead is gone and we are back to about 12 > seconds again.
Another option would be to read all the files in as a single bloc. What about doing something like this; replace whatever loads all the .d files, maybe something like this: include $(wildcard *.d) with something like this instead: _ := $(shell cat *.d > $(TMP)/all.d) include $(TMP)/all.d I haven't tried this but it should work, I think. > Are there any ideas (or opinions) about how to get rid of this type > of undesirable overhead? One of our own ideas is to provide a special > variant of -include (called e.g. dinclude?), intended to include just > .d files, without maintaining MAKEFILE_LIST. On the other hand it > might be feasible to improve the efficiency of appending to a > variable. Any suggestions? My preference would be to allow MAKEFILE_LIST to be "special" such that it's updated when expanded, rather than being updated as files are read in. That way the value will always be accurate but the price for computing the string won't be paid until/unless the variable is expanded. Of course, we'd still need to maintain an ordered list of pointers to files that will be added to MAKEFILE_LIST when it's requested, but we already need to maintain file structures for each of those so that won't be too much overhead. The tricky part is creating an infrastructure for these types of variables. There are other similar ones, such as .VARIABLES, which could benefit from similar treatment. We have a flag that marks variables that behave specially when set; this would be a flag that marks variables that behave specially when expanded. _______________________________________________ Bug-make mailing list Bug-make@gnu.org https://lists.gnu.org/mailman/listinfo/bug-make
