On Fri, May 17, 2019 at 1:05 PM Sebastian Huber < sebastian.hu...@embedded-brains.de> wrote:
> On 20/03/2019 13:16, Sebastian Huber wrote: > > Hello, > > > > the RTEMS project has a covoar tool: > > > > https://git.rtems.org/rtems-tools/tree/tester/covoar > > > > I have seen some reports generated by it, but I never used it myself. > > I have trouble to find out how it works. There is nothing in the > > documentation: > > > > cd rtems-docs > > grep -ri covoar . > > > > There is some documentation available here: > > > > https://devel.rtems.org/wiki/GCI/Documentation/CoverageAnalysis/Coverage > > > > If I invoke the rtems-test like this in the erc32 BSP build directory, > > I get an error: > > > > rtems-test --rtems-bsp=erc32-sis --rtems-tools=/build/rtems/5 > > --coverage --target=sparc-rtems5 . > > RTEMS Testing - Tester, 5.0.not_released > > Command Line: /build/rtems/5/bin/rtems-test --rtems-bsp=erc32-sis > > --rtems-tools=/build/rtems/5 --coverage --target=sparc-rtems5 . > > Python: 3.6.5 (default, Mar 31 2018, 19:45:04) [GCC] > > Host: Linux-4.12.14-lp150.12.48-default-x86_64-with-glibc2.3.4 (Linux > > huber-nb-linux.suse 4.12.14-lp150.12.48-default #1 SMP Tue Feb 12 > > 14:01:48 UTC 2019 (268f014) x86_64 x86_64) > > Traceback (most recent call last): > > File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in > > <module> > > test.run(sys.argv[1:], command_path = base) > > File "/build/rtems/5/share/rtems/tester/rt/test.py", line 300, in run > > trace = cov_trace) > > File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 363, > > in __init__ > > self.target = self.macros['target'] > > File "/build/rtems/5/share/rtems/rtemstoolkit/macros.py", line 181, > > in __getitem__ > > raise IndexError('key: %s' % (key)) > > IndexError: key: target > > > > I made some progress to try it out myself. The next obstacle was this: > > rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov > --coverage --no-clean --log=leon3-test.log > sparc-rtems5/c/leon3/testsuites/samples/hello.exe > RTEMS Testing - Tester, 5.0.not_released > [1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe > Passed: 1 > Failed: 0 > User Input: 0 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 0 > Wrong Version: 0 > Wrong Build: 0 > Wrong Tools: 0 > ---------------- > Total: 1 > Average test time: 0:00:01.505144 > Testing time : 0:00:01.505144 > error: symbol parser write failed for score > > To figure out what went wrong, I removed the try/except block from > tester/rt/coverage.py and got: > > rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov > --coverage --no-clean --log=leon3-test.log > sparc-rtems5/c/leon3/testsuites/samples/hello.exe > RTEMS Testing - Tester, 5.0.not_released > [1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe > Passed: 1 > Failed: 0 > User Input: 0 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 0 > Timeout: 0 > Invalid: 0 > Wrong Version: 0 > Wrong Build: 0 > Wrong Tools: 0 > ---------------- > Total: 1 > Average test time: 0:00:01.504073 > Testing time : 0:00:01.504073 > Traceback (most recent call last): > File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in > <module> > test.run(sys.argv[1:], command_path = base) > File "/build/rtems/5/share/rtems/tester/rt/test.py", line 387, in run > coverage_runner.run() > File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 401, in > run > parser.write_ini(sset) > File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 293, in > write_ini > object_files = [o for o in os.listdir(self.symbol_sets[sset]) if > o[-1] == 'o'] > FileNotFoundError: [Errno 2] No such file or directory: > '/sparc-rtems5/c/leon3/cpukit/score/src' > > This is more useful error output from my point of view. It seems you > have to use an absolute path for the test directory. > > rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov > --coverage --no-clean --log=leon3-test.log > /build/git-build/b-leon3/sparc-rtems5/c/leon3/testsuites/samples/hello.exe > [....] > Coverage generating reports > error: copying tree: /build/rtems/5/share/rtems/tester/covoar/covoar.css > -> /build/git-build/b-leon3/leon3-sis-coverage/score/covoar.css: [Errno > 2] No such file or directory: > '/build/rtems/5/share/rtems/tester/covoar/covoar.css' > > -rtems-tools also needs an absolute path. > I copied some files by hand. A full test run ended up in: > > rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov > --coverage --no-clean --log=leon3-test.log > /build/git-build/b-leon3/sparc-rtems5/c/leon3/testsuites > RTEMS Testing - Tester, 5.0.not_released > [ 3/710] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | > sparc/leon3-sis: ada_nsecs.exe > [...] > [710/710] p:684 f:5 u:6 e:0 I:0 B:3 t:0 i:2 W:0 | > sparc/leon3-sis: tmttest01.exe > Passed: 692 > Failed: 6 > User Input: 6 > Expected Fail: 0 > Indeterminate: 0 > Benchmark: 3 > Timeout: 1 > Invalid: 2 > Wrong Version: 0 > Wrong Build: 0 > Wrong Tools: 0 > ------------------ > Total: 710 > Average test time: 0:00:00.427096 > Testing time : 0:05:03.237825 > > Running coverage analysis: score > (/build/git-build/b-leon3/leon3-sis-coverage/score) > Coverage time: 0:17:53.695293 > > Running coverage analysis: rtems > (/build/git-build/b-leon3/leon3-sis-coverage/rtems) > Coverage time: 0:12:55.898577 > > Running coverage analysis: sapi > (/build/git-build/b-leon3/leon3-sis-coverage/sapi) > error: coverage: covoar failure:: 10 > > This is actually a bug in covoar or a bug somewhere in spversion01.exe We have a ticket for this https://devel.rtems.org/ticket/3515. To track it down, I manually ran covoar with the trace files using the following command ( note: you need --no-clean option in rtems-test for this) covoar -S /home/lunatic/development/rtems/rtems-tools/tester/rtems/testing/coverage/leon3-sis-symbols.ini -O coverage -E/home/lunatic/development/rtems/rtems-tools/tester/rtems/testing/coverage/Explanations.txt -p RTEMS-5 sparc-rtems5/c/leon3/testsuites/sptests/spversion01.exe -v -fTSIM ==== . . . Created unified coverage map for rtems_initialize_executive (0x0 - 0x4b) Created unified coverage map for rtems_shutdown_executive (0x0 - 0x13) Created unified coverage map for rtems_fatal_error_occurred (0x0 - 0x13) Created unified coverage map for rtems_get_version_string (0x0 - 0xb) Created unified coverage map for _IO_Initialize_all_drivers (0x0 - 0x4b) Created unified coverage map for rtems_io_initialize (0x0 - 0x5b) Created unified coverage map for rtems_version (0x0 - 0xb) Created unified coverage map for rtems_version_major (0x0 - 0x7) Created unified coverage map for rtems_version_minor (0x0 - 0x7) error: ExecutableInfo::findCoverageMap: rtems_version_revision ==== So there's something in rtems_version_revision() in spversion01/ that's making it unhappy. > -- > Sebastian Huber, embedded brains GmbH > > Address : Dornierstr. 4, D-82178 Puchheim, Germany > Phone : +49 89 189 47 41-16 > Fax : +49 89 189 47 41-09 > E-Mail : sebastian.hu...@embedded-brains.de > PGP : Public key available on request. > > Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG. > > _______________________________________________ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
