OK I tried to setup the JDK coverage, and this is NOT easy. Right now I'm blocked by a bug in the Jenkins Jacoco plugin: https://issues.jenkins-ci.org/browse/JENKINS-36168 Need to solve it first. If I find how to setup JDK coverage for an Ant project, it will deserve serious documentation. I will probably update JOSM wiki, but maybe it could land somewhere on https://wiki.openjdk.java.net/display/quality/Quality+Outreach for other projects?
2017-11-06 23:51 GMT+01:00 Alexandre (Shura) Iline < [email protected]>: > > On Nov 6, 2017, at 2:48 PM, Vincent Privat <[email protected]> > wrote: > > Sure! Currently coverage is restricted to JOSM classes, for obvious > performance reasons, as we run a full build after each commit. > But I can add a parameter to change the monitored packages in our EA > builds. > Is it OK if I add java, javax, sun and com.sun ? Or do you prefer to > restrict only to packages from java.desktop module? > > > Thank you! > > Frankly, java.desktop is what I am more interested myself right now. > > java, javax, sun and com.sun would also cover what I am asking for, of > course. > > Shura > > > > > 2017-11-06 19:31 GMT+01:00 Alexandre (Shura) Iline < > [email protected]>: > >> This is great that you are touching the new builds, thank you, Vincent. >> >> I am not familiar with JOSM code, but the app is great! I have played >> with it in the past. >> >> Do you think it would be possible, in addition to what you are doing, to >> collect code coverage for the JDK code? Specifically for the now existing >> java.desktop module. I do not know much of JaCoCo - we are using JCov, as >> you can imagine - so I do not know how easy it is. If it is easy, can you >> flip the switch? >> >> Shura >> > On Nov 3, 2017, at 8:33 AM, Vincent Privat <[email protected]> >> wrote: >> > >> > Hello Rory, >> > Thank you :) >> > >> > Here is how we test various OpenJDK builds: >> > 1.1: we use only EA builds from http://jdk.java.net/ >> > 2.1.1: no >> > 2.1.2/2.1.3: we run all unit and integration tests of JOSM >> > 2.1.4: yes, using jacoco >> > 2.2: all tests are automated >> > 2.3: it depends. During OpenJDK9 cycle we did not run a test series for >> each weekly build, but rather once every two or three builds, unless we >> were waiting for a specific correction from OpenJDK or another tool. At the >> end of development, we were not waiting for any correction from OpenJDK but >> were eagerly waiting for the updates of ASM and everything that as blocked >> by the non-availability of ASM (in particular various Maven plugins) >> > 2.4: Yes, they are visible here: >> > - JOSM core: https://josm.openstreetmap.de/ >> jenkins/job/Java-EarlyAccess-JOSM/ >> > - JOSM community plugins: https://josm.openstreetmap.de/ >> jenkins/job/Java-EarlyAccess-JOSM-Plugins/ >> > >> > I just have initiated JDK10 tests today, and we have 53 failures caused >> by Byte Buddy not recognizing Java version 10: >> > >> > java.lang.IllegalArgumentException: Unknown Java version: 10 >> > at nl.jqno.equalsverifier.internal.lib.bytebuddy.ClassFileVersi >> on.ofJavaVersion(ClassFileVersion.java:135) >> > >> > >> > I'm sure Rafael is aware of it. I will see how to update our tools >> (equalsverifier here). >> > >> > Cheers, >> > Vincent >> > >> > 2017-11-03 14:05 GMT+01:00 Rory O'Donnell <[email protected]>: >> > >> > Hi Vincent, >> > >> > I noticed that JOSM have been commended in the last two Quality reports >> [1] [2] for the number and >> > indeed the quality of bugs you have found and logged against various >> early access releases. >> > Thanks again for your contribution to the quality of our releases! >> > >> > The big question is how do you do it ? >> > >> > We would love to hear more details about your test environment, what >> tests you run etc. Below is a >> > series of questions that come to mind, but there may be other details >> that you would like to share. >> > • Builds >> > • do you build your own OpenJDK builds, use Oracle's or >> other EA builds ? >> > • Tests >> > • What tests do you run >> > • do you run the JCK ? >> > • open regression tests ? >> > • other tests ? >> > • do you measure code coverage ? >> > • Are the tests Automated or Manual ? >> > • How frequently to do execute the tests >> > • Are the results visible via an external link >> > Rgds, Rory >> > >> > [1] https://wiki.openjdk.java.net/display/quality/Quality+Outrea >> ch+report+September+2017 >> > [2] https://wiki.openjdk.java.net/display/quality/Quality+Outrea >> ch+report+March+2017 >> > -- >> > Rgds,Rory O'Donnell >> > Quality Engineering Manager >> > Oracle EMEA , Dublin, Ireland >> > >> > >> >> > >
