Understand GCC test process
Hi, I'm needing to analyze the execution flow of a test, but don't understand how the test drivers, e.g., gcc-dg.exp or dg.exp, access the source code of GCC. When a test starts, what is the first function that is called? How can I know that? On another note, I observed the execution traces of some tests from gcc.dg testsuite and I realized that more than 60% of the called functions are common to all traces of all testes. So, I would like to confirm if that happens because the drive is the same (dg.exp)? or if there is a common process across all tests. Does anyone can clarify the test process of GCC? Thanks, Sabrina Souto
Re: Understand GCC test process
>> Hi, >> I'm needing to analyze the execution flow of a test, but don't >> understand how the test drivers, e.g., gcc-dg.exp or dg.exp, access >> the source code of GCC. When a test starts, what is the first function >> that is called? How can I know that? > > The first function is main(), because the tests just run the compiler. > DejaGnu doesn't access the source code of GCC, it just runs the > compiler executable. OK, I understand. But when a test run the compiler, all compiler stages/phases are always executed? how the test knows the compiler stage must be run? from the "dg-do" actions (preprocess, compile, assemble, link, run)? how the "dg-do" actions are processed/performed? Thanks, -- Sabrina Souto
Re: Understand GCC test process
I was seeing these files but I could not put the puzzle pieces
together in my mind, and after you explained, all pieces make sense
now. Thanks for the explanation, Jonathan.
The testing process is clear now, but I still not understanding what
can explain that amount of common functions (over 60%) across all
traces of the tests (that I analyzed) from gcc.dg test suite, could
you help me with this? I have some hypotheses:
(1) All tests run the compiler and there is a large "core" code that
is executed regardless of the compiler option?
(2) Since all tests run the same driver - dg.exp, I could say that
this great amount of common functions is due to what is specified in
the driver, and the remaining few different functions are related with
the particularity of each test?
(3) None of those. Do you have another explanation?
Thanks,
On Wed, Oct 7, 2015 at 10:22 AM, Jonathan Wakely wrote:
> On 7 October 2015 at 14:03, Sabrina Souto wrote:
>>>> Hi,
>>>> I'm needing to analyze the execution flow of a test, but don't
>>>> understand how the test drivers, e.g., gcc-dg.exp or dg.exp, access
>>>> the source code of GCC. When a test starts, what is the first function
>>>> that is called? How can I know that?
>>>
>>> The first function is main(), because the tests just run the compiler.
>>> DejaGnu doesn't access the source code of GCC, it just runs the
>>> compiler executable.
>>
>> OK, I understand. But when a test run the compiler, all compiler
>> stages/phases are always executed?
>
> No, only the stages requested by the dg-do directive.
>
>> how the test knows the compiler
>> stage must be run? from the "dg-do" actions (preprocess, compile,
>> assemble, link, run)? how the "dg-do" actions are processed/performed?
>
> That's initially handled by DejaGnu, not GCC. See the site-wide dg.exp
> file (on my system that is /usr/share/dejagnu/dg.exp) which says:
>
> # dg-do do-what-keyword [{ target/xfail selector }]
> # `do-what-keyword' is tool specific and is passed unchanged to
> # ${tool}-dg-test. An example is gcc where `keyword' can be any of:
> # preprocess|compile|assemble|link|run
> # and will do one of: produce a .i, produce a .s, produce a .o,
> # produce an a.out, or produce an a.out and run it (the default is
> # compile).
>
> So that passes the argument to the callback gcc-dg-test, which is
> defined in gcc/testsuite/lib/gcc-dg.exp, and that sets the
> 'compile_type' variable to one of assemble, preprocess, link etc. and
> sets the appropriate output filename for the action.
>
> That is then translated into command-line options for the compiler, so
> a 'compile' test will be run with -c, an 'assemble' test will be run
> with -S, a 'link' test will have no special option (so all stages run)
> and a 'run' test does the same as 'link' but also runs the executable.
--
Sabrina Souto
Re: Understand GCC test process
> What exactly are you tracing, and how? I'm proposing an approach for testing configurable system in my research, and I'm trying to apply it to GCC. So, I instrumented the GCC function calls (in the first level of ..gcc-version-x.x/gcc/ and the dirs related to C/C++) and some options, in a semi-automatic way (I tried to use PIN, but it was very slow...). > Are you only tracing the 'gcc' driver? Or every process and child > process that gets spawned by 'make'? I'm tracing the 'gcc' driver. > > Typically to run the testsuite you run 'make', which runs DejaGnu's > 'runtest' shell script, which runs the 'expect' program (written in > Tcl), which invokes the 'gcc' driver, which invokes the actual > compiler, 'cc1', to compile a testcase. So a lot of that is common to > every testcase. Do you mean, for example, that the compiler will always build the AST, translate to intermediary code representations, etc., regardless of the compilation option... and it corresponds to a lot of common code across all tests? Thanks, Sabrina Souto
Re: Understand GCC test process
> OK, all that does is process some options and then call another
> executable, which does the actual compilation.
>
> To compile C code it calls cc1, to compile C++ code it calls cc1plus etc.
>
> So if you're only tracing the 'gcc' driver then you are basically only
> tracing the code for parsing command-line arguments and choosing which
> compiler to execute.
>
>
>>>
>>> Typically to run the testsuite you run 'make', which runs DejaGnu's
>>> 'runtest' shell script, which runs the 'expect' program (written in
>>> Tcl), which invokes the 'gcc' driver, which invokes the actual
>>> compiler, 'cc1', to compile a testcase. So a lot of that is common to
>>> every testcase.
I'm understanding that the major of code that I instrumented (in the
first level of ..gcc-version-x.x/gcc/ and the dirs related to C/C++)
is part of the 'gcc' driver. If so, this explains the great amount of
common function calls across the tests.
>> Do you mean, for example, that the compiler will always build the AST,
>> translate to intermediary code representations, etc., regardless of
>> the compilation option... and it corresponds to a lot of common code
>> across all tests?
>
> No, for example if it is run with -E it will only do preprocessing.
I ran
make RUNTESTFLAGS='dg.exp=c90-float-1.c -v -v' check-gcc
And I saw in the log:
...
doing compile
Invoking the compiler as
../gcc-r227092/objdir/gcc/testsuite/g++/../../xg++ -B/...
...
The test ../testsuite/gcc.dg/c90-float-1.c contains the action: /* {
dg-do preprocess } */
So, why "doing compile" was in the execution log? I thought that the
compiler would not be called in this case.
Am I running the test in a wrong way?
>
> But you're not tracing the compiler anyway. The 'gcc' executable is
> not the compiler.
I think I understood.
But, how can I differentiate between the 'gcc' driver's code and the
compiler's code?
All the code inside ..gcc-version-x.x/gcc/ corresponds to the 'gcc'
driver's code?
Where can I find all the code that implements the compiler and
preprocessor? In libcpp, libcc1, boehm-gc ?
Please, help me.
Thanks,
--
Sabrina Souto
Re: Understand GCC test process
> See also > https://gcc.gnu.org/ml/gcc-patches/2015-06/msg00765.html - a proposal to add > unit tests to GCC. Grate work! I'll also try to use this framework. > Probably the message is wrong. Nevertheless, IIRC, the preprocessor is also > run > as xg++. xg++ will invoke the compiler proper (cc1plus), which will perform > preprocessing. Good to know. > Mostly in gcc and libcpp directories. Code from include and libiberty is also > used. > Consider looking at https://gcc.gnu.org/onlinedocs/gccint/Source-Tree.html and > gcc/Makefile.in in more detail. > > Finally, note that GCC has some means of test coverage analysis: you can > configure/build it with "--disable-bootstrap --enable-coverage", run the tests > and then use gcov. Helpful explanation, Mikhail. Thank you.
Crowdsourcing - Help with experiment using GCC
Hi everybody, I am working on my PhD in a project that uses crowdsourcing [1, 2, 3] to extract rules/relationships among options of configurable systems from documentation, and one of the systems that I am studying is GCC. This is an invitation to participate in an activity of crowdsourcing within this project. The time estimated for this activity is 3 minutes per task. If you are interested in participating, please visit the link below and inform both the number of tasks that you're interested in and your e-mail: http://goo.gl/forms/bo2e5Q3G4j The link below shows an example of the task and a possible answer: http://www.cin.ufpe.br/~sfs/survey/taskexample.html [1] http://www.merriam-webster.com/dictionary/crowdsourcing [2] http://jis.sagepub.com/content/38/2/189 [3] http://searchcio.techtarget.com/definition/crowdsourcing Thank you, -- Sabrina de F. Souto. http://www.cin.ufpe.br/~sfs
