uschindler opened a new pull request, #12188: URL: https://github.com/apache/lucene/pull/12188
This is an altenative version to PR #12042. The problem with the previous PR (and also the Java 19 version) is mainly the following issues: - We need to add a toolkit entry for each Java version we want to support. As Panama-Foreign is not yet finished, any developer who wants to build Lucene needs to either download Java 19+20+21 to actually compile the JARs or the autodownloader of Gradle will do it. - We need to patch the class files to remove the preview bits after compilation The approach here is a new idea that came to me when I studied, how the `--release` flag is implemented in javac. It works like that: Java has basically a copy of all class files of previous Java versions in some specially designed JAR file. When you request `--release 11` from Java compiler, it will overlay the compile classpath wit this JAR file that contains all classes (only signatures). Of course this will spend a lot of space, so there are 2 optimizations: it removes all bytecode from those JAR files, so all class files in it have empty method bodies. In addition, it will only add modified class files. After each relaese of Java they build this special signatures JAR file and store it in the JDK folder. FYI, preview API are excluded there, this is why we can't compile. The PR here implements the following: - It stores a similar (but simplified) version of the relevant classes in a "panama-foreign-jdk19.apijar" and "panama-foreign-jdk20.apijar" - The apijar files are regenerated by the "regenerate task" or by calling `gradlew :lucene:core:generatePanamaForeignApiJar19` and/or `gradlew :lucene:core:generatePanamaForeignApiJar20`. This task uses Gradle's toolkit API to launch the correct Java version. So only people who regenerate files need to have Java 19, 20, 21,... installed. For Java 20 at moment you need it really installed locally, as it is impossible to autodownload it. - When generating the APIJAR (a zip file in standard JAR format) it uses ASM to load the class files from the current JVM using the small tool `ExtractForeignAPI.java` which is executed using the toolkit API. It will then strip all bytecode and also strips the `@PreviewFeature` annotation (this allows to compile against the class file without Java adding the preview flags to compilation result. The outputted class files also get the version number for Java 11 class files. It also removes the option for nesthosts (not needed). - When we compile out Java 19, Java 20, Java 21 MR-JAR, it will compile our sourceSet with the normal Java 11 or Java 17 compiler, but supplies `--patch-module java.base=panama-foreign-jdk20.apijar`. It also open the new foreign package, so it gets visible. This PR allows anybody to compile the MR-JAR classes without the JDK installed. We can also merge this PR now, as the JDK 20 release is already in RC phase, so the API is finalized. The APIJAR files don't get outdated anymore. We can also release Lucene now with JDK 20 support! Downsides: - We have 2 or 3 binary files in out Git checkout (they are each approx 20 Kilobytes). An alternative would be to dump a txt-only signatures file, but that complicates it really. We can do that for licensing purposes. - Licensing is hard, this is the main reason why I asked for comments: Is it ok to have a JAR file with method signatures in our source checkout? There's no code included (bytecode stripped)! The license is the GNU GPL with Classpath extension, but we compile against that already- If anybody has an idea, speak up. If we agree, this would be the easiest to support MR-JARs of MMapDirectory in future (also with Java 21 where it is not yet fully sure that the API gets to its final stage! ....unfortunately.... damn! The same approach could be used for vector APIs in future. Please add a comment about if you agree to do it like that! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
