On Sun, Apr 2, 2017 at 12:33 PM, Tanu Hari Dixit <tokencol...@gmail.com> wrote:
> Hi Andy, > > Joel and Gedare pointed me to the following: > > https://devel.rtems.org/wiki/Developer/Projects/Open/ > PythonCoverageReporting > https://devel.rtems.org/ticket/2261 > http://kmiesowicz.blogspot.in/p/esa-socis-2014.html > > Last link is to the work done by a previous student. > Hope this helps. > My proposal does not take covoar into consideration. Though, I have > planned to produce xml reports from rtems-tester that can be > integrated into coverage testing. Also, I have planned to use this xml > data to get plots (for visualizing results) as a stretch goal (you can > look in the "Future Improvements" Section in my proposal). > covoar directly produces the html and text reports we have now. My recollection is that it doesn't have to be touched and that the trick is invoking it on a more focused basis than Core vs Developmental. So invoke covoar from rtems-tester mulitple times on a single run of the tests rather than twice at most. Personally, I think Core is still a good description for the core of the tasking (e.g. score, rtems, sapi, and posix) and that each directory other than that should be individual. But that's my marketing view. I could easily be argued that for consistency, it should all be on a per directory basis. At some point, we might discuss covoar writing a neutral format and using Python to do more but that is pure conjecture. I have no idea what the intermediate format would be. Writing Sphinx output for reports would be desirable from covoar. :) --joel > > Regards, > Tanu Hari Dixit. > > On Sun, Apr 2, 2017 at 6:50 PM, Gedare Bloom <ged...@rtems.org> wrote: > > Joel's e-mail client messed up my reply thread. ;) Andy, have a read > > through https://gedare.github.io/pdf/FelShe15A.pdf if you haven't > > found it already. > > > > Getting qemu-couverture to build via RSB and then integrating the > > coverage scripts into rtems-test, would be a good start to a project. > > There is a student (Tanu)oar dir who has proposed rtems-test > improvements, so > > you two would need to coordinate efforts for this piece to avoid too > > much duplicated work/waste. Improvements to the reports etc are also > > welcome, and then of course using the reports to guide code > > refactoring to improve coverage is somewhat open-ended. > > > > -Gedare > > > > On Sat, Apr 1, 2017 at 9:34 PM, Joel Sherrill <j...@rtems.org> wrote: > >> > >> > >> On Apr 1, 2017 7:59 PM, "Andy MacGregor" <amacgregor.2...@comcast.net> > >> wrote: > >> > >> On 03/31/2017 09:52 PM, Gedare Bloom wrote: > >> > >> Don't forget the deadline is Monday to submit your Final > >> PDF and proof of enrollment. The "Final" PDF can be submitted multiple > >> times, so go ahead and submit it when you finish a first draft. Add > >> your draft proposal to the tracking table at > >> https://devel.rtems.org/wiki/GSoC/2017 also, to get some possible > >> feedback. > >> > >> > >> Thanks for the welcome! (I realize I am late to the GSOC application > table). > >> > >> I'm set on improving code coverage of the testsuite, but I found some > >> questions as I'm starting to look into how its done. > >> > >> The link on the wiki to the current code status is currently broken, so > I > >> tried to generate a sample coverage report for 4.12 using ./do_coverage, > >> ./run_coverage and ./coverage_cron as suggested on the wiki here but > without > >> any luck. Is there an up to date coverage report available? > >> I modified the VERSIONS-COVERAGE file like the wiki said, and once that > >> didn't work dug around in the script but fix it myself quickly. > >> > >> Could updating these coverage analysis scripts be a viable component of > a > >> GSOC proposal? > >> > >> > >> Yes. In fact, the idea was to rework the coverage scripts in shell to be > >> part of the rtems-tester package and in Python. There is some work on > this > >> by a previous student which you can find or one of us can dig out. > Starting > >> with it, making it work across multiple BSPs, independent of any hard > coded > >> paths, improving the output (we did not get far enough that I recall > >> worrying about the quality of the output), etc could be a fair amount of > >> work. > >> > >> Also we have privately discussed switching the RSB qemu upstream source > to > >> GNATCoverage at GitHub. This is a coverage and embedded focused > downstream > >> which includes the coverage trace output I used for the qemu support in > >> covoar. I'm > >> > >> In addition, I'd like to actually make some improvements the code > coverage > >> itself. > >> The most recent information I could find was these bar graphs from 2009. > >> If these graphs reflect the current coverage state, I think that > improving > >> coverage for the i386/pc386, > >> would be interesting because there is room for improvement in both the > >> Baseline and Developmental groups (if these coverage statistics are up > to > >> date). > >> > >> > >> It doesn't matter which BSP you choose for coverage since the code and > tests > >> are the same. > >> > >> This also reminds me of the other improvements we had wanted. We wanted > to > >> switch from Baseline and developmental to a more directory oriented > approach > >> for the reports. For example, the reports should be at a granularity of > say > >> the Dos file system or shell as reports. The way reports are generated > now > >> it results in a skewed view when a large body of code is added to the > >> developmental set. Adding the Dos filesystem resulted in a 15-20% drop > in > >> the coverage reported in the Developmental set. That is a bad way to > report > >> and misleading. It detracts from the directories which have near 100% > >> coverage. > >> > >> > >> Does anyone know of a good reason to choose another BSP over i386/pc386? > >> > >> > >> Users fly the SPARC leon3 and not the pc386. :) > >> > >> > >> I've just posted a draft of my proposal to the GSOC page. > >> If there's any feedback I'll be sure to update my proposal in time. > >> > >> > >> Dinner time here. Hopefully my feedback makes sense and you can > incorporate > >> it. > >> > >> > >> > >> > >> -Andy MacGregor > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> _______________________________________________ > >> devel mailing list > >> devel@rtems.org > >> http://lists.rtems.org/mailman/listinfo/devel > >> > >> > >> > >> _______________________________________________ > >> devel mailing list > >> devel@rtems.org > >> http://lists.rtems.org/mailman/listinfo/devel > > _______________________________________________ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel >
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
