gortiz opened a new pull request, #10528: URL: https://github.com/apache/pinot/pull/10528
#9162 describes why Apache Pinot cannot run in modern Java versions. The main reason is that LArray, the library we use to map more than 2^31-1 bytes, calls some private methods that are not available in modern Java versions. This method includes: - A new SPI in the form of `PinotBufferFactory`. This factory is used to change the way buffers are created in Pinot. - A modification of the current tests we have that test `PinotDataBuffer` in order to be able to replicate the same tests with other implementations. - A default implementation of `PinotBufferFactory` that uses LArray. - A default implementation of `PinotBufferFactory` that tries to use ByteBuffers when possible and otherwise delegate on another factory. - A new implementation of `PinotDataBuffer` that uses Unsafe. This implementation should work in Java 11, 17 and 20. - A new implementation of `PinotBufferFactory` that uses the Unsafe `PinotDataBuffer`. - A new implementation of `PinitDataBuffer` that uses Foreign Memory api as it is defined in Java 20. I've executed the `PinotDataBuffer` tests in Java 17 and 20 in my personal Linux machine and they work. I still need to try them in Mac. I've also launched a QuickStart with Java 20 trying to find some problems. I did not executed all other tests. Assuming the new code is correct, what we would need to add before this is merged is all the plumbing required to run all tests with Java 17 and 20. This is theoretical not very complex, but it is time consuming. In order to run the Unsafe code with Java 17 and 20 we need to add the following JVM arguments at test and runtime: ``` --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED ``` In order to run with Foreign Memory, we need to add the following JVM arguments both at compile, test and runtime: ``` --enable-preview ``` As far as I know it is not necessary to add the previous -add-opens. - feature? bugfix? - performance? - Draft -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
