Hi everyone, Looking into our way to Java 17, I wanted to share with the community findings/thoughts and align on course of action.
We already deprecated scripted UDFs so we can remove them when the time to switch from Java8&11 to Java 11&17 comes. I removed the ant script tasks and created custom ant tasks to workaround the need of Nashorn. Ant is also upgraded now on trunk. With this Cassandra trunk compiles with warnings about the Security manager being deprecated and other security deprecation warnings(I mention it for awareness here). I pushed to my personal docker hub account a version of our testing image that has Java 17 installed and I worked on the build file, shell scripts and config to push testing with Java 17 to CircleCI. To just start Cassandra out of the box on Java 17 we also need as a least minimum to add the following, further to what we already open in Java 11: *--add-opens java.base/sun.nio.ch <http://sun.nio.ch/>=ALL-UNNAMED* *--add-opens java.base/java.io <http://java.io/>=ALL-UNNAMED* *--add-opens java.base/java.util.concurrent=ALL-UNNAMED* *--add-opens java.base/java.util=ALL-UNNAMED* *--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED* *--add-opens java.base/java.nio=ALL-UNNAMED* *--add-opens java.base/java.lang=ALL-UNNAMED* *--add-exports java.base/sun.nio.ch <http://sun.nio.ch/>=ALL-UNNAMED* *--add-exports java.base/java.io <http://java.io/>=ALL-UNNAMED* *--add-exports java.base/java.util.concurrent=ALL-UNNAMED* *--add-exports java.base/java.util=ALL-UNNAMED* *--add-exports java.base/java.util.concurrent.atomic=ALL-UNNAMED* *--add-exports java.base/java.nio=ALL-UNNAMED* *--add-exports java.base/java.lang=ALL-UNNAMED* This is a quick run of *jdeps -jdkinternals --multi-release 17 apache-cassandra-4.1-SNAPSHOT.jar.*: * apache-cassandra-4.1-SNAPSHOT.jar -> java.rmi* * apache-cassandra-4.1-SNAPSHOT.jar -> jdk.unsupported* * org.apache.cassandra.io.util.Memory -> sun.misc.Unsafe JDK internal API (jdk.unsupported)* * org.apache.cassandra.utils.FastByteOperations$UnsafeOperations -> sun.misc.Unsafe JDK internal API (jdk.unsupported)* * org.apache.cassandra.utils.FastByteOperations$UnsafeOperations$1 -> sun.misc.Unsafe JDK internal API (jdk.unsupported)* * org.apache.cassandra.utils.JMXServerUtils$JmxRegistry -> sun.rmi.registry.RegistryImpl JDK internal API (java.rmi)* * org.apache.cassandra.utils.memory.MemoryUtil -> sun.misc.Unsafe JDK internal API (jdk.unsupported)* *Warning: JDK internal APIs are unsupported and private to JDK implementation that are* *subject to be removed or changed incompatibly and could break your application.* *Please modify your code to eliminate dependence on any JDK internal APIs.* *For the most recent update on JDK internal API replacements, please check: * *https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool <https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool>* Also a quick workaround I applied for test purposes in order to be able to run the in-jvm tests can be seen here[4], and I assume something like that is also needed for the simulator and other places which I went ahead and changed in order just to unblock the preliminary testing so we can get the full picture. To be revised later. This was the issue we were hitting: java.lang.RuntimeException: java.lang.NoSuchFieldException: modifiers at org.apache.cassandra.distributed.impl.Instance.lambda$startup$11(Instance.java:691) at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:81) at org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47) at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: java.lang.NoSuchFieldException: modifiers at java.base/java.lang.Class.getDeclaredField(Class.java:2610) at org.apache.cassandra.net.Verb.unsafeSetSerializer(Verb.java:366) at org.apache.cassandra.distributed.impl.Instance.lambda$startup$11(Instance.java:612) To fire testing I used some config suggested by Jonathan Shook which he used for testing with Java 16 some time ago[1]. We will need to tune it, those were used to do preliminary work and initial investigations only. One important topic I want to bring to the community's attention is our dependency management. [2] is a discussion we had a long time ago when I joined the project. So is it correct to say for Java 17 we don't update anything if there is no immediate need (and this considers only trunk of course)? In my branch I updated bytebuddy, asm, chronicle queues, byteman, mockito, jacoco, ecj, sjk so far. Based on feedback from CASSANDRA-17392 I tried to update netty and run our tests with Java 17 and I ran into *java.lang.ClassCastException: class org.apache.cassandra.utils.memory.BufferPool$Chunk cannot be cast to class sun.nio.ch.DirectBuffer* . I didn't have the chance to investigate it but it is on our table so good to be mentioned. Even with the update of Chronicle queues[3] we still have to add for FQLtool and AuditLogging: --add-opens=java.base/java.lang.reflect=ALL-UNNAMED \ --add-exports=java.base/jdk.internal.ref=ALL-UNNAMED \ --add-exports=java.base/sun.nio.ch=ALL-UNNAMED \ --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED \ --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \ --add-opens=jdk.compiler/com.sun.tools.javac=ALL-UNNAMED \ --add-opens=java.base/java.lang=ALL-UNNAMED \ --add-opens=java.base/java.lang.reflect=ALL-UNNAMED \ --add-opens=java.base/java.io=ALL-UNNAMED \ --add-opens=java.base/java.util=ALL-UNNAMED\ After that, from what I saw there is only one type of failure related to those tools to be investigated that cannot be fixed directly with some other internals opening. I also updated Jolokia in the DTest repo which seemed to solve any Jolokia related problems, at least our tests are not failing due to Jolokia related problems from what I saw so far but officially I see Jolokia builds running on JDK 8, 9, and 11. I didn't find any info around Java 17 and Jolokia so far. These are a few of the things we hit now, so I wanted to mention in order to give you an idea of the direction the things are going. So I am interested to hear any comments, particularly on our approach to dependencies updates as a project. Also, how much are we willing to extend the usage of --add-opens and --add-exports at this moment? The Java community recommendation is this to be our last option. We have some dependencies which still do not officially support Java 17 or which are not even fully tested with Java 17. Some of them will work out of the box, some will need more internals to be opened maybe or some other actions to be taken, I still don't know the answers to all failures I see and sometimes I fix one but another one appears. There are moving parts and I also didn't go ahead and update all dependencies based on the previous discussions. (I also saw that when Java 11 was introduced we didn't update all dependencies?) All of this is only to be able to bring our CI to some stable condition on Java 17 as a starting point and get an idea of what is needed. Also, I guess soon we can join the OpenJDK Quality Outreach program[5] suggested before as we get more details that can be already discussed. I can go ahead and do it if no one is against it, the contact can be just our dev-mailing list I guess. [1]https://jaxenter.com/apache-cassandra-java-174575.html [2]https://lists.apache.org/thread/l1ch2ffcnv2m5l9tp9mpvgzwlvx667sd [3]https://chronicle.software/chronicle-support-java-17/ [4] https://github.com/apache/cassandra/commit/4daf0b149bc1524dfb2212ab958f4d45feb79577 [5]https://wiki.openjdk.java.net/display/quality/Quality+Outreach Ekaterina Dimitrova
