On 03/04/2017 03:53, Joel Sherrill wrote:
On Sun, Apr 2, 2017 at 12:33 PM, Tanu Hari Dixit <tokencol...@gmail.com
<mailto:tokencol...@gmail.com>> wrote:
Hi Andy,
Joel and Gedare pointed me to the following:
https://devel.rtems.org/wiki/Developer/Projects/Open/PythonCoverageReporting
<https://devel.rtems.org/wiki/Developer/Projects/Open/PythonCoverageReporting>
https://devel.rtems.org/ticket/2261
<https://devel.rtems.org/ticket/2261>
http://kmiesowicz.blogspot.in/p/esa-socis-2014.html
<http://kmiesowicz.blogspot.in/p/esa-socis-2014.html>
Last link is to the work done by a previous student.
Hope this helps.
My proposal does not take covoar into consideration. Though, I have
planned to produce xml reports from rtems-tester that can be
integrated into coverage testing. Also, I have planned to use this xml
data to get plots (for visualizing results) as a stretch goal (you can
look in the "Future Improvements" Section in my proposal).
covoar directly produces the html and text reports we have now. My
recollection is that it doesn't have to be touched and that the trick
is invoking it on a more focused basis than Core vs Developmental.
I think covoar does need some refactoring. There are things we can do
now the tool resides in the rtems-tools repo that makes the tool
portable, more stable, faster and better.
First is the use of external tools being invoked via the `system` libc
call to collect information that can be done directly via the
rtemstoolkit. The makes the tool portable because the toolkit runs on
the Windows and Unix and it is based on standards based file formats, ie
ELF. As an example to highlight changes that need to happen take this
function:
https://git.rtems.org/rtems-tools/tree/tester/covoar/ObjdumpProcessor.cc#n233
a) The symbols data can be loaded directly from the ELF file and it is
quick and fully portable. See
https://git.rtems.org/rtems-tools/tree/linkers/rtems-syms.cpp#n463 as an
example.
b) The `system` call is not portable, eg Windows, because the command
expects a Unix shell, pipes and `sed`. We have a requirement to support
Windows. See
https://git.rtems.org/rtems-tools/tree/rtemstoolkit/rld-process.h#n235
for executing on all hosts.
c) Are the generated files cleaned up on exceptions, crashes etc. See
https://git.rtems.org/rtems-tools/tree/rtemstoolkit/rld-process.h#n54
and the process based support to handle temporary files and output.
I also see the running of the simulator as the function of `rtems-test`
as it handles the jobs and manage the state of the simulator and any
files created. It would hand the trace data to be processed to covoar
and get back the report, ie XML data, then clean up.
So invoke covoar from rtems-tester mulitple times on a single run
of the tests rather than twice at most.
Sorry, I do not understand this. Is rtems-tester the repo or is this
rtems-test the command?
Personally, I think Core is still a good description for the core of
the tasking (e.g. score, rtems, sapi, and posix) and that each
directory other than that should be individual. But that's my marketing
view. I could easily be argued that for consistency, it should all
be on a per directory basis.
I wonder if ELF notes would help by letting us set an attribute or a few
attributes for each function and this guides how we handle them. It
would be nice if these are implemented close to the actual function's
source. As we have direct access to the ELF file we can extract these
notes and so the attributes and we can then directly map what is
happening. For example we can have attributes (ie EFL notes) such as
`rtems/score/cpu`, `rtems/score/threads`, and `rtems/score/mutex` etc
and you could say cover `rtems` or `rtems/score` etc. And if we also set
the standard the code conforms to such as `pthread_create` with the
standard could we end up with a coverage map based on the standard?
Using ELF notes means we do not have to manage the process by the
position of the source in the source tree.
At some point, we might discuss covoar writing a neutral format
and using Python to do more but that is pure conjecture.
I think this is real and important and not conjecture. The tool
generating HTML directly conflicts with the proposed parent wrapper,
`rtems-test`. The `rtems-test` tool's requirement is to export data in a
standard format like XML that can be imported in to a reporting tool to
generate HTML etc. A user can then decide if the HTML, ReST, SQL or
whatever meets their needs or they can take the XML and incorporate it
into their own workflow that meets their requirements. I would like to
see covar reports be XML and writing XML does not need a special
library, it is just print statements. Have a look at the RSB's XML
writer Sebastian added,
https://git.rtems.org/rtems-source-builder/tree/source-builder/sb/reports.py#n417.
We can use python to import the XML and generate the reports as it comes
with suitable XML support.
I have
no idea what the intermediate format would be. Writing Sphinx
output for reports would be desirable from covoar. :)
XML.
Chris
_______________________________________________
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel
