shangeyao opened a new pull request, #4373: URL: https://github.com/apache/streampark/pull/4373
## Summary - Parse Spark version from `$SPARK_HOME/jars/spark-core_*.jar` and `RELEASE` before falling back to `spark-submit --version`, so Spark 4.x environments can be registered while StreamPark Console runs on JDK 8. - Add `SparkEnvUtils` to auto-resolve `JAVA_HOME` from `spark-env.sh`, process environment, or system detection, and apply it in Yarn Spark job submission via `SparkLauncher.setJavaHome()`. - Document Spark JDK requirements and manual configuration scenarios in `script/SPARK_JDK_GUIDE.md` / `SPARK_JDK_GUIDE.zh.md`, with UI hints on the Spark Home form. ## Test plan - [x] `JAVA_HOME=$(/usr/libexec/java_home -v 1.8) ./mvnw test -pl streampark-common -Dtest=SparkVersionTest,SparkEnvUtilsTest -Dspotless.check.skip=true` - [ ] Register Spark 4.1.2 in Console with StreamPark running on JDK 8 - [ ] Submit a Spark 4.x job on Yarn and verify `JAVA_HOME` is picked up correctly - [ ] Verify Spark 2.x / 3.x environment registration still works --- **AI Disclosure** - Model: Claude Opus 4.6 - Platform/Tool: Cursor - Human Oversight: partially reviewed - Prompt Summary: Fix Spark 4.1.2 registration failure caused by JDK mismatch, make version detection transparent to users, and document when manual JDK configuration is required. Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
