lldb and ccache
Bugzilla #20297

There have been several unsuccessful attempts by BSI to use ccache to speed up lldb unit tests.  When using ccache, the unit tests fail.  A common failure mode when using ccache with the lldb unit test framework, is the inability to set a breakpoint at the first instruction in main().

The unit test framework is rather complicated and fragile, so to try to see what was going on, i created a simple “Hello World” program outside of the unit test framework to investigate the problem.

	#include <stdio.h>

	int main(int argc, char const *argv[])
	{
		printf("Hello\n");
		return 0;
	}

Step 1
As a baseline, i first created the inferior executable using clang with the compiler options used by the unit test framework
	clang -g -O0 -m64 main.cpp

Loading the inferior executable into lldb, i was able to set a breakpoint as expected with
	br set -l 5

Step 2
Next, i created the inferior executable with ccache
	ccache clang -g -O0 -m64 main.cpp

The executable was the same size as Step 1 and i was able to set a breakpoint.

Step 3
While digging into what happens in the unit test framework, i noticed that the unit tests are created in two phases - a compile-only phase and a link-only phase.  To model this, i created my test case with
	clang -g -O0 -m64 -c -o main.o main.cpp
	clang -g -O0 -m64 main.o

This creates an executable the same size as Step 1 and i was able to set a breakpoint.

Step 4
The next thing i tried was to use ccache for the two-phase method.
	ccache clang -g -O0 -m64 -c -o main.o main.cpp
	ccache clang -g -O0 -m64 main.o

This creates an executable with a different size than any of the other executables and
	br set -l 5
doesn’t work and returns “Breakpoint 1: no locations (pending).”

Step 5
I noticed that the unlinked object file (main.o) was a different size when using ccache.  objdump shows that the code is the same, but that some of the data is different.

I tried linking without ccache
	ccache clang -g -O0 -m64 -c -o main.o main.cpp
	clang -g -O0 -m64 main.o

But that still creates an executable with an unusual size and the breakpoint still can’t be set.

Step 6
I tried debugging the different executables with gdb.  i was able to set a gdb breakpoint with all of the different flavors of executables using
	break main.cpp:5

Conclusion(s)
lldb seems to have a problem reading object files generated by ccache, if ccache was used with a compiler in a compile-only phase.  gdb does not seem to have this problem, so there is probably nothing fundamentally wrong with the object file generated by ccache.

It may be possible to enable ccache for the unit tests by rewriting the lldb unit test framework to create unit test executables in a compound compile/link operation, but this would just hide lldb’s problem with ccache object files and would not help if lldb itself was created with ccache in two phases and needed to be debugged or if some lldb user was using lldb to debug some application using ccache and created with two phases.