Joel's e-mail client messed up my reply thread. ;) Andy, have a read through https://gedare.github.io/pdf/FelShe15A.pdf if you haven't found it already.
Getting qemu-couverture to build via RSB and then integrating the coverage scripts into rtems-test, would be a good start to a project. There is a student (Tanu) who has proposed rtems-test improvements, so you two would need to coordinate efforts for this piece to avoid too much duplicated work/waste. Improvements to the reports etc are also welcome, and then of course using the reports to guide code refactoring to improve coverage is somewhat open-ended. -Gedare On Sat, Apr 1, 2017 at 9:34 PM, Joel Sherrill <j...@rtems.org> wrote: > > > On Apr 1, 2017 7:59 PM, "Andy MacGregor" <amacgregor.2...@comcast.net> > wrote: > > On 03/31/2017 09:52 PM, Gedare Bloom wrote: > > Don't forget the deadline is Monday to submit your Final > PDF and proof of enrollment. The "Final" PDF can be submitted multiple > times, so go ahead and submit it when you finish a first draft. Add > your draft proposal to the tracking table at > https://devel.rtems.org/wiki/GSoC/2017 also, to get some possible > feedback. > > > Thanks for the welcome! (I realize I am late to the GSOC application table). > > I'm set on improving code coverage of the testsuite, but I found some > questions as I'm starting to look into how its done. > > The link on the wiki to the current code status is currently broken, so I > tried to generate a sample coverage report for 4.12 using ./do_coverage, > ./run_coverage and ./coverage_cron as suggested on the wiki here but without > any luck. Is there an up to date coverage report available? > I modified the VERSIONS-COVERAGE file like the wiki said, and once that > didn't work dug around in the script but fix it myself quickly. > > Could updating these coverage analysis scripts be a viable component of a > GSOC proposal? > > > Yes. In fact, the idea was to rework the coverage scripts in shell to be > part of the rtems-tester package and in Python. There is some work on this > by a previous student which you can find or one of us can dig out. Starting > with it, making it work across multiple BSPs, independent of any hard coded > paths, improving the output (we did not get far enough that I recall > worrying about the quality of the output), etc could be a fair amount of > work. > > Also we have privately discussed switching the RSB qemu upstream source to > GNATCoverage at GitHub. This is a coverage and embedded focused downstream > which includes the coverage trace output I used for the qemu support in > covoar. I'm > > In addition, I'd like to actually make some improvements the code coverage > itself. > The most recent information I could find was these bar graphs from 2009. > If these graphs reflect the current coverage state, I think that improving > coverage for the i386/pc386, > would be interesting because there is room for improvement in both the > Baseline and Developmental groups (if these coverage statistics are up to > date). > > > It doesn't matter which BSP you choose for coverage since the code and tests > are the same. > > This also reminds me of the other improvements we had wanted. We wanted to > switch from Baseline and developmental to a more directory oriented approach > for the reports. For example, the reports should be at a granularity of say > the Dos file system or shell as reports. The way reports are generated now > it results in a skewed view when a large body of code is added to the > developmental set. Adding the Dos filesystem resulted in a 15-20% drop in > the coverage reported in the Developmental set. That is a bad way to report > and misleading. It detracts from the directories which have near 100% > coverage. > > > Does anyone know of a good reason to choose another BSP over i386/pc386? > > > Users fly the SPARC leon3 and not the pc386. :) > > > I've just posted a draft of my proposal to the GSOC page. > If there's any feedback I'll be sure to update my proposal in time. > > > Dinner time here. Hopefully my feedback makes sense and you can incorporate > it. > > > > > -Andy MacGregor > > > > > > > > > > > _______________________________________________ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel > > > > _______________________________________________ > devel mailing list > devel@rtems.org > http://lists.rtems.org/mailman/listinfo/devel _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
