Hello Jerzy,
Actually, the same configuration can be adapted to run in SIS (which is what we
use most of the times). Inside the QDP you have the file
qual-tool/config-variants/sparc-gr712rc-smp-user-qual.yml. Change the file to
the following (you can copy and paste the content, replacing the current one):
build-directory: build-sparc-gr712rc-smp-user-qual
post-process-items:
- uid: /dirs/djf-svr-deploy/dir
path: /directory
action: set
value: ${/variant:/deployment-directory}/user_doc/djf/svr
- uid: /dirs/ddf-sdd-deploy/dir
path: /directory
action: set
value: ${/variant:/deployment-directory}/user_doc/ddf/sdd
- uid: /steps/build-djf-svr
path: /config-file
action: set
value: rtems/djf/svr/config_user.yml
- uid: /package-build
path: /links
action: set
value:
- role: build-step
uid: steps/build-bsp-qual-only
- role: build-step
uid: steps/build-bsp-qual-only-coverage
- role: build-step
uid: steps/run-local-target-qual-only
- role: build-step
uid: steps/run-local-target-qual-only-coverage
- role: build-step
uid: steps/build-ddf-sdd
- role: build-step
uid: steps/build-djf-svr
spec-paths:
- spec-spec
- spec-glossary
- config
Then follow the steps as described in section “11.3 Guidance for RTEMS
Qualification in User’s Environment” to generate your own SVR. After you have
this working example, it would be easier to see how the things work. The
scripts/configuration files in the qual-tool folder could be adapted to what
you want (i.e: just generate coverage for your tests). Feel free to ask me if
you have any difficulty!
Best regards
José
From: Jerzy Jaskuc <jask...@tcd.ie>
Sent: 16 de março de 2022 19:16
To: VALDEZ Jose <jose.val...@edisoft.pt>
Cc: devel@rtems.org
Subject: Re: Support for test coverage analysis in RTEMS 6
Hi José,
I'm aware of the QDP, although it is pretty big so I got confused and only
followed `11.3 Guidance for RTEMS Qualification in User’s Environment`, which
didn't work as I assume it needs actual board for tests to not be`Invalid`
The reports generated are exactly what I'm looking for and steps you mentioned
seems much more manageable!
I can see gcno files and I can see gcda bytes within the logs of the test run.
I will take a look at adapting the python scripts then.
Thanks a lot for your help!
All the best,
Jerzy
On Wed, 16 Mar 2022 at 17:05, VALDEZ Jose
<jose.val...@edisoft.pt<mailto:jose.val...@edisoft.pt>> wrote:
Hello Jerzy,
Recently we concluded a project, the RTEMS SMP Qualification which also needed
to gather coverage for RTEMS (which had the participation of Andrew Butterfield
from TCD/LERO). I suggest that you take a look at
https://rtems-qual.io.esa.int/, download one of the QDPs (maybe you want the
GR712RC, SMP) and look for the docs/djf/svr document. There you may find the
coverage folder, with the html coverage reports and in the svr.pdf the coverage
summary table.
If this is what you want for your test, I suggest to take a look at
qual-tool/svrmanager.py and qual-tool/ccoverage_process.py.
Basically for this what you need to do:
* Compile RTEMS with coverage flags. This will generate the gcno files
* Run your test, the reports should generate the gcda bytes (Sebastian Huber
added support for coverage in RTEMS itself)
* Use/adapt ccoverage_process.py to generate the coverage reports, which makes
use of the gcovr tool.
Hope this helps
José
-----Original Message-----
From: devel <devel-boun...@rtems.org<mailto:devel-boun...@rtems.org>> On Behalf
Of Jerzy Jaskuc
Sent: 16 de março de 2022 16:46
To: devel@rtems.org<mailto:devel@rtems.org>
Subject: Support for test coverage analysis in RTEMS 6
Hi,
I've been looking into generating code coverage reports from RTEMS tests in
RTEMS 6.
I'm using SPARC with gr712rc BSP.
I'm following the steps outlined in the coverage analysis documentation
https://docs.rtems.org/branches/master/user/testing/coverage.html
However, when I run it on any tests, even sample/hello.exe I get the logs in
lines of the following:
`RTEMS Testing - Tester, 6.0.not_released Command Line: ./rtems-test
--rtems-tools=/home/yoman/RTEMS/rtems/6
--log=log_leon3_sis --rtems-bsp=leon3-sis-cov --coverage
/home/yoman/RTEMS/src/rtems/build/sparc/gr712rc/testsuites/validation/ts-validation-0.exe
Host: Linux ubuntu 5.13.0-35-generic #40~20.04.1-Ubuntu SMP Mon Mar 7
09:18:32 UTC 2022 x86_64
Python: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
Host: Linux-5.13.0-35-generic-x86_64-with-glibc2.29 (Linux ubuntu
5.13.0-35-generic #40~20.04.1-Ubuntu SMP Mon Mar 7 09:18:32 UTC 2022 x86_64
x86_64)
[1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 L:0 i:0 W:0 | sparc/leon3-sis:
ts-validation-0.exe
Passed: 1
Failed: 0
User Input: 0
Expected Fail: 0
Indeterminate: 0
Benchmark: 0
Timeout: 0
Test too long: 0
Invalid: 0
Wrong Version: 0
Wrong Build: 0
Wrong Tools: 0
Wrong Header: 0
----------------
Total: 1
Average test time: 0:00:02.008333
Testing time : 0:00:02.008333
Running coverage analysis (/home/yoman/RTEMS/rtems/6/bin/leon3-sis-coverage)
Function is both external and inlined: _Message_queue_Create Function is both
external and inlined: _Message_queue_Create Function is both external and
inlined: _Message_queue_Create Function is both external and inlined:
_Thread_Initialize_information Function is both external and inlined:
_Thread_Initialize_information Function is both external and inlined:
_Thread_queue_Initialize Function is both external and inlined:
_Thread_queue_Initialize Function is both external and inlined:
_Thread_queue_Resume Function is both external and inlined:
_Thread_queue_FIFO_first Function is both external and inlined:
_Thread_queue_FIFO_first Function is both external and inlined:
_Thread_CPU_budget_consume_and_yield
Function is both external and inlined: _Thread_CPU_budget_consume_and_yield
Function is both external and inlined: _Thread_Join Function is both external
and inlined: _Thread_Join Function is both external and inlined:
_Thread_Set_state_locked Function is both external and inlined:
_Thread_Set_state_locked Function is both external and inlined:
_Thread_Set_state_locked Function is both external and inlined:
_Thread_Set_state_locked Function is both external and inlined:
_Timecounter_Binuptime Function is both external and inlined:
_Timecounter_Bintime Function is both external and inlined:
_Timecounter_Getboottimebin Function is both external and inlined:
_Thread_Priority_perform_actions Function is both external and inlined:
_Thread_Priority_perform_actions Function is both external and inlined:
_Thread_Clear_state_locked Function is both external and inlined:
_Thread_Clear_state_locked Function is both external and inlined:
_Thread_Clear_state_locked Function is both external and inlined:
_Thread_Clear_state_locked Function is both external and inlined:
_Malloc_System_state Function is both external and inlined:
_Malloc_System_state Function is both external and inlined:
_Malloc_System_state Function is both external and inlined:
rtems_counter_initialize_converter
Function is both external and inlined: rtems_counter_initialize_converter
Function is both external and inlined: rtems_version_minor Function is both
external and inlined: _Heap_Get_first_and_last_block Function is both external
and inlined: _Heap_Get_first_and_last_block Error while running
sparc-rtems6-objdump on
/home/yoman/RTEMS/src/rtems/build/sparc/gr712rc/testsuites/validation/ts-validation-0.exe
Objdump error in generating objdump
Coverage time: 0:00:01.085511
Coverage generating reports
Coverage analysis finished: /home/yoman/RTEMS/rtems/6/bin`
I've tried to do a fresh VM fresh RTEMS build, results are exactly the same.
It seems like it's something to do with `objdump`? That would explain the
generated report containing only unreferenced symbols.
Any tips on what's wrong or where to start looking to try and fix it?
I wanted to generate coverage reports for the tests I developed, but not sure
if there's any other way than through making this part work.
Thanks and all the best!
Jerzy
_______________________________________________
devel mailing list
devel@rtems.org<mailto:devel@rtems.org>
http://lists.rtems.org/mailman/listinfo/devel
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
