Greetings,
The company I work for has a large code base consisting of many components. As
part of the build process we compute the dependency graph for these components
and load it into GNU make. We've noticed that processing these dependencies
can take a significant amount of time in some cases. For instance, on a
Windows 7 build machine it takes over 30 minutes.
During our investigation we found that make 3.82 performs more stat() calls
than 3.81. In our Windows build environment we're using a SMB2 based filer and
these additional stat() calls result in network traffic to the filer. We're
unsure why the stat() calls aren't being satisfied by the local file cache, but
that's a separate issue.
Here's a simple example of what we're seeing. Consider the following test
Makefile.
build : $(addsuffix .ts,a b c)
%.ts :
touch $@
a.ts :
b.ts : a.ts
c.ts : b.ts a.ts
I build the Makefile the first time to create the *.ts files. After that I use
strace to execute make 3.81 on Linux. The *.ts file already exist so there
should be nothing to do. Here are the stat() calls for my targets.
stat("a.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("b.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("c.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
For this test make 3.81 performs 1 stat() call per target file. Next I use
strace to execute make 3.82 on Linux. Again, the *.ts files already exist.
Here are the stat() calls for my targets.
stat("./a.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./b.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./c.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./%.ts", 0x7fffc59db4c0) = -1 ENOENT (No such file or directory)
stat("./a.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./b.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./a.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./c.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./b.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("./a.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("a.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("b.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat("c.ts", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
For this test it appears that make 3.82 performs N stat() calls per file, where
N is based on the number of times each file is mentioned as a target or a
dependency. In our system, that means 1 stat() versus thousands of stat()
calls per file.
Is this difference in behavior between make 3.81 and 3.82 intentional? If so,
is there anything we could do to reduce the number of stat() calls being made?
The additional stat() calls don't have much of an impact on our NFS based Linux
build environment, but they do have a noticeable impact on our SMB2 based
Windows build environment.
Thanks.
--Troy Runkel
_______________________________________________
Bug-make mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/bug-make
