yuboxx commented on code in PR #4198: URL: https://github.com/apache/datafusion-comet/pull/4198#discussion_r3178795524
########## docs/source/contributor-guide/development.md: ########## @@ -276,6 +276,78 @@ Comet uses generated source files that are too large for IntelliJ's default size by going to `Help -> Edit Custom Properties...`. For example, adding `idea.max.intellisense.filesize=16384` increases the file size limit to 16 MB. +#### Working with Spark 4.x profiles in IntelliJ IDEA + +Spark 4.x requires JDK 17 and Scala 2.13. When switching an existing IntelliJ project from the +default Spark 3.x setup to a Spark 4.x profile, it is usually best to re-import the Maven project +from a clean IntelliJ configuration: + +1. Close the IntelliJ project. +2. Delete the `.idea` directory from the repository root. +3. Make sure `protoc` is available. On macOS, install it with `brew install protobuf`; on Linux, + install `protobuf-compiler` or the equivalent package for your distribution. +4. Make sure `JAVA_HOME` points to JDK 17. On macOS, you can select it with: + + ```sh + export JAVA_HOME=$(/usr/libexec/java_home -v 17) + export PATH="$JAVA_HOME/bin:$PATH" + ``` + + On Apple Silicon, the selected JDK architecture must match the native Rust target. Verify that + `file "$JAVA_HOME/lib/server/libjvm.dylib"` reports `arm64`. + +5. Build the profile once from the command line so generated sources and native artifacts are in + place: + + ```sh + PROFILES="-Pspark-4.0" make release + ``` + + The `spark-4.0` profile sets Scala 2.13 and Java 17 properties. If you need to be explicit, use + `PROFILES="-Pspark-4.0 -Pscala-2.13 -Pjdk17" make release`. + + The Maven profile is named `jdk17` in this project. + + If the native build previously used a different JDK, clear Cargo's cached JNI link path before + rebuilding: + + ```sh + cd native && cargo clean -p hdfs-sys --release Review Comment: Somehow I ran into link issues with only that repo in my test, `cargo clean` could work as well and maybe more clean. Updated the md file, feel free to take a another look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
