On 22/07/2024 23:33, Christopher Schultz wrote:
Mark,
On 7/22/24 12:53, Mark Thomas wrote:
All,
Today I have configured the tomcat-tck repository to run the EL,
Servlet, Pages and WebSocket TCKs once every day for all combinations
of JDK 17 & 21, Ubuntu latest, MacOS latest and Windows latest using
GitHub actions.
There were a few issues to iron out but these should now all be resolved.
The TCK will run at just after 08.00 UTC every day and it will use the
latest Tomcat 11 SNAPSHOT (these are updated on every commit by
buildbot).
Windows seems to take a little longer than the others but the full TCK
run (all four TCKs) is complete in just under 25 minutes. Considering
it used to take longer than that to run any of the old TCKs, kudos to
the Jakarta EE folks that have been working on the refactoring.
Tomcat 11.0.x currently passes the TCK (as it should).
I have no plans to formally certify Tomcat as passing the TCK over and
above what I have already completed as part of the release process for
each of the specifications (the specification release process requires
at least one compatible implementation).
Nice work.
My guess is that getting Tomcat to be formally-certified would take (1)
money (2) politics and (3) other stuff nobody wants to deal with any
time soon.
For certification the bar is pretty low. It would probably take longer
on the Eclipse side getting the certified version added to the various
websites than it would be on the ASF side doing the actual certification.
Whether even that low level of effort is something we want to do is TBD.
There is very little (no?) demand from users for formal certification.
The real value to me is in running and passing the tests. Therefore,
I'll probably use the tomcat-tck repo to test each release candidate as
it is published and include those results in my vote. I haven't figured
out how to automate that but I am thinking about it.
If we want to use the "Jakarta EE compatible" logo then that is where we
hit your points 1, 2 and 3 in spades.
We'd need to sign a trademark agreement and my reading of that is that
the ASF does not currently have the right membership of Eclipse to be
able to use the logo. Fixing that looks likely to take some time and
politics to resolve - particularly since the ASF is unlikely to want to
pay for am Eclipse membership.
Given that we are free to make factual statements such as "Tomcat 11.0.x
passes the latest Annotations, EL, Pages, Servlet and WebSocket TCKs" or
"Tomcat 11.0.0-M20 is a compatible implementation of the Jakarta Servlet
6.0 specification" I'm not at all convinced of the need to use a logo.
I wonder if we made it sooooo easy to certify (e.g. automated builds and
automated TCK executions) that someone else might just do it for us (I'm
looking at you, /Eclipse Foundation/.. I seem to remember a conference
presentation about how important Tomcat was to Eclipse).
It is easy enough for anyone to perform the certification.
When the TCK runs, does it produce a report which includes the
identities of the files that were used to run produce that report?
Specifically, I'm wondering if the TCK report can be used to verify that
a *specific release* has passed and TCK and that can be verified by an
external observer.
No. But all the information required is there in the Maven build. It
probably needs a simple(ish) Maven plugin to generate it.
I'm not volunteering to add TCK runs as part of the standard release
process but if "anyone" could produce a TCK report which shows the
entirely-reproducible build that is Tomcat x.y.z is what has been
tested, that would be really great.
It should only take a couple of minutes to generate the required
information manually from one of the GitHub action runs.
Instead of just "trust me, I ran it on my machine and this report says
it passed", anyone could verify that the same artifacts we released were
the ones the TCK was run on.
They can do that now. One of the benefits of getting the TCK running as
a GitHub action was that it forced me to fix all places I wasn't aware
of that the tomcat-tck repo was depending on things I already had
installed on my dev machine.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
