Thanks Joel for the correction. Regards, Tanu Hari Dixit.
On Sun, Apr 2, 2017 at 11:23 PM, Joel Sherrill <j...@rtems.org> wrote: > > > On Sun, Apr 2, 2017 at 12:33 PM, Tanu Hari Dixit <tokencol...@gmail.com> > wrote: >> >> Hi Andy, >> >> Joel and Gedare pointed me to the following: >> >> >> https://devel.rtems.org/wiki/Developer/Projects/Open/PythonCoverageReporting >> https://devel.rtems.org/ticket/2261 >> http://kmiesowicz.blogspot.in/p/esa-socis-2014.html >> >> Last link is to the work done by a previous student. >> Hope this helps. >> My proposal does not take covoar into consideration. Though, I have >> planned to produce xml reports from rtems-tester that can be >> integrated into coverage testing. Also, I have planned to use this xml >> data to get plots (for visualizing results) as a stretch goal (you can >> look in the "Future Improvements" Section in my proposal). > > > covoar directly produces the html and text reports we have now. My > recollection is that it doesn't have to be touched and that the trick > is invoking it on a more focused basis than Core vs Developmental. > So invoke covoar from rtems-tester mulitple times on a single run > of the tests rather than twice at most. > > Personally, I think Core is still a good description for the core of > the tasking (e.g. score, rtems, sapi, and posix) and that each > directory other than that should be individual. But that's my marketing > view. I could easily be argued that for consistency, it should all > be on a per directory basis. > > At some point, we might discuss covoar writing a neutral format > and using Python to do more but that is pure conjecture. I have > no idea what the intermediate format would be. Writing Sphinx > output for reports would be desirable from covoar. :) > > --joel > >> >> >> Regards, >> Tanu Hari Dixit. >> >> On Sun, Apr 2, 2017 at 6:50 PM, Gedare Bloom <ged...@rtems.org> wrote: >> > Joel's e-mail client messed up my reply thread. ;) Andy, have a read >> > through https://gedare.github.io/pdf/FelShe15A.pdf if you haven't >> > found it already. >> > >> > Getting qemu-couverture to build via RSB and then integrating the >> > coverage scripts into rtems-test, would be a good start to a project. >> > There is a student (Tanu)oar dir who has proposed rtems-test >> > improvements, so >> >> > you two would need to coordinate efforts for this piece to avoid too >> > much duplicated work/waste. Improvements to the reports etc are also >> > welcome, and then of course using the reports to guide code >> > refactoring to improve coverage is somewhat open-ended. >> > >> > -Gedare >> > >> > On Sat, Apr 1, 2017 at 9:34 PM, Joel Sherrill <j...@rtems.org> wrote: >> >> >> >> >> >> On Apr 1, 2017 7:59 PM, "Andy MacGregor" <amacgregor.2...@comcast.net> >> >> wrote: >> >> >> >> On 03/31/2017 09:52 PM, Gedare Bloom wrote: >> >> >> >> Don't forget the deadline is Monday to submit your Final >> >> PDF and proof of enrollment. The "Final" PDF can be submitted multiple >> >> times, so go ahead and submit it when you finish a first draft. Add >> >> your draft proposal to the tracking table at >> >> https://devel.rtems.org/wiki/GSoC/2017 also, to get some possible >> >> feedback. >> >> >> >> >> >> Thanks for the welcome! (I realize I am late to the GSOC application >> >> table). >> >> >> >> I'm set on improving code coverage of the testsuite, but I found some >> >> questions as I'm starting to look into how its done. >> >> >> >> The link on the wiki to the current code status is currently broken, so >> >> I >> >> tried to generate a sample coverage report for 4.12 using >> >> ./do_coverage, >> >> ./run_coverage and ./coverage_cron as suggested on the wiki here but >> >> without >> >> any luck. Is there an up to date coverage report available? >> >> I modified the VERSIONS-COVERAGE file like the wiki said, and once that >> >> didn't work dug around in the script but fix it myself quickly. >> >> >> >> Could updating these coverage analysis scripts be a viable component of >> >> a >> >> GSOC proposal? >> >> >> >> >> >> Yes. In fact, the idea was to rework the coverage scripts in shell to >> >> be >> >> part of the rtems-tester package and in Python. There is some work on >> >> this >> >> by a previous student which you can find or one of us can dig out. >> >> Starting >> >> with it, making it work across multiple BSPs, independent of any hard >> >> coded >> >> paths, improving the output (we did not get far enough that I recall >> >> worrying about the quality of the output), etc could be a fair amount >> >> of >> >> work. >> >> >> >> Also we have privately discussed switching the RSB qemu upstream source >> >> to >> >> GNATCoverage at GitHub. This is a coverage and embedded focused >> >> downstream >> >> which includes the coverage trace output I used for the qemu support in >> >> covoar. I'm >> >> >> >> In addition, I'd like to actually make some improvements the code >> >> coverage >> >> itself. >> >> The most recent information I could find was these bar graphs from >> >> 2009. >> >> If these graphs reflect the current coverage state, I think that >> >> improving >> >> coverage for the i386/pc386, >> >> would be interesting because there is room for improvement in both the >> >> Baseline and Developmental groups (if these coverage statistics are up >> >> to >> >> date). >> >> >> >> >> >> It doesn't matter which BSP you choose for coverage since the code and >> >> tests >> >> are the same. >> >> >> >> This also reminds me of the other improvements we had wanted. We wanted >> >> to >> >> switch from Baseline and developmental to a more directory oriented >> >> approach >> >> for the reports. For example, the reports should be at a granularity of >> >> say >> >> the Dos file system or shell as reports. The way reports are generated >> >> now >> >> it results in a skewed view when a large body of code is added to the >> >> developmental set. Adding the Dos filesystem resulted in a 15-20% drop >> >> in >> >> the coverage reported in the Developmental set. That is a bad way to >> >> report >> >> and misleading. It detracts from the directories which have near 100% >> >> coverage. >> >> >> >> >> >> Does anyone know of a good reason to choose another BSP over >> >> i386/pc386? >> >> >> >> >> >> Users fly the SPARC leon3 and not the pc386. :) >> >> >> >> >> >> I've just posted a draft of my proposal to the GSOC page. >> >> If there's any feedback I'll be sure to update my proposal in time. >> >> >> >> >> >> Dinner time here. Hopefully my feedback makes sense and you can >> >> incorporate >> >> it. >> >> >> >> >> >> >> >> >> >> -Andy MacGregor >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> devel mailing list >> >> devel@rtems.org >> >> http://lists.rtems.org/mailman/listinfo/devel >> >> >> >> >> >> >> >> _______________________________________________ >> >> devel mailing list >> >> devel@rtems.org >> >> http://lists.rtems.org/mailman/listinfo/devel >> > _______________________________________________ >> > devel mailing list >> > devel@rtems.org >> > http://lists.rtems.org/mailman/listinfo/devel > > _______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel
