gortiz opened a new issue, #15343: URL: https://github.com/apache/pinot/issues/15343
Since 1.0.0, Pinot dropped support for Java 8 and since then the minimum source and runtime Java version is 11. Here I propose increasing that minimum to Java 21 for Pinot 1.4.0. *This proposal doesn't affect the Pinot driver, which will still support Java 8*. # Current state Pinot officially supports Java 11 and 21. At least, these are the versions we test. Docker images use Java 11, 17 and 21. The default docker image (the one tagged as `latest`) is Java 11. A Java program compiled with Java 11 should be able to run on Java 11, 17, 21, 25, etc without changes as long as it only uses public APIs. That is not the case for Pinot, which uses some internal APIs that are not available in Java 11. These APIs are mainly used to get access to low level features like network or memory management. Specifically, we had some issues to run in Java 21 given we were using some internal APIs through LArray to get access to offheap buffers larger than 2GBs. These issues were fixed in https://github.com/apache/pinot/issues/11656 and since Sept 2023 (https://github.com/apache/pinot/pull/11672) all Pinot tests are executed in both Java 11 and 21. # Java versions Java 8, 11, 17, 21 are LTS versions, which officially means that Oracle supports them for longer time that non-LTS versions. In practice, other vendors like AdoptOpenJDK, Amazon, Azul, BellSoft, IBM, Red Hat, and others follows the same model and support versions marked by Oracle as LTS for a longer time. Java 11 is the first LTS version after Java 8, and it is the minimum version for many projects. The situation with Java 11 is getting similar to Python 2.7, where many projects are stuck with it, and it is hard to move forward. This seems to be changing slowly. Oracle and RedHat stopped supporting Java 11 in 2023 and 2024, respectively and popular libraries/frameworks like Spring Framework 6, Hibernate 6 or Micronaut 4 require Java 17. Oracle support for Java 17 will end in 2026 and RedHat will support it until 2027. Java 21 is the latest LTS version, and it is supported by Oracle until 2028 and by RedHat until 2029. Java 25 is the next LTS release. It will be released in September 2025 and supported by Oracle until 2030 and by RedHat until 2031. # Source vs runtime versions The source version is the minimum Java version required to compile the code. In Java this version is also the minimum runtime version. Since Pinot 1.0.0 we are using Java 11 as the minimum source and runtime version although the recommended runtime version is Java 21 given it provides better performance and more runtime features like new GCs. # Cons of the update Increasing the minimum Java version would have the inconvenience that users would need to install a use a newer Java version. This shouldn't be a problem given that Java 21 is already supported by all vendors and in fact Java 11 is not supported by most of them, but we know that some users move slowly to newer versions. That is the reason why even we can run Pinot in Java 21, we are still using 11 as the source version. # Pros of the update The main reason to increase the minimum Java version is to be able to use new features and improvements in the Java language and also runtime. Although we can get some of the Java 21 performance improvements by running Pinot in Java 21, we can't use of some of them. Things we cannot use go from nice to have programming features but also to important performance improvements, including: - Code level improvements: - `var` keyword (https://openjdk.java.net/jeps/286) - `instanceof` pattern matching (https://openjdk.java.net/jeps/394), - `switch` expressions (https://openjdk.java.net/jeps/361) - `sealed` classes (https://openjdk.org/jeps/409) - `record` classes (https://openjdk.java.net/jeps/395) - Performance improvements - Virtual threads (https://openjdk.org/jeps/444), which could be useful for MSE. - Foreign memory access (https://openjdk.org/jeps/442), which could be useful and faster than our offheap memory access API. # Proposal 1. Open this issue to discuss how to proceed. 2. Get feedback from the community trying to find out if there are users that would be affected by this change. 3. If the community agrees, we will update the minimum Java version to Java 21. This change may be done for Pinot 1.4.0 or for a future release. # Alternatives ## Upgrade to Java 17 first Although Java 17 is also LTS, it is going to be EOL for Oracle and RedHat in 2026 and 2027, respectively. It also doesn't include any of the performance improvements listed above. We are also running tests in Java 21, so we are confident that Pinot works in Java 21. We could consider to upgrade to Java 17 as a first step to make the transition easier for our users, but we would need to upgrade to Java 21 in the near future. ## Keep Java 11 as the minimum version and ignore new features We could keep Java 11 as the minimum version and consider upgrading to Java 21 in a future release. This would allow users to keep using Java 11 for a longer time, but we would not be able to use new features and some performance improvements in Java 21. ## Keep Java 11 as the minimum version and use plugins for new features We could keep Java 11 as the minimum version and use plugins to provide new features. This means developers won't be able to use new features in the Java language, but we could provide some of the performance improvements by using plugins. The main disadvantage of this approach is that it would require more work from developers. It is not easy to maintain, in the same repository, a plugin that requires a different Java version than the main code. This was in fact the main blocker of https://github.com/apache/pinot/issues/12809 and the reason why we decided to stop working on it. The corollary of this is that it is an incentive to create forks or separate projects which may not be open source to add support for these new features, which is not ideal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
