Repository: spark Updated Branches: refs/heads/branch-2.0 df4f87106 -> ef8d16ec4
[SPARK-15451][BUILD] Use jdk7's rt.jar when available. This helps with preventing jdk8-specific calls being checked in, because PR builders are running the compiler with the wrong settings. If the JAVA_7_HOME env variable is set, assume it points at a jdk7 and use its rt.jar when invoking javac. For zinc, just run it with jdk7, and disable it when building jdk8-specific code. A big note for sbt usage: adding the bootstrap options forces sbt to fork the compiler, and that disables incremental compilation. That means that it's really not convenient to use for normal development, but should be ok for automated builds. Tested with JAVA_HOME=jdk8 and JAVA_7_HOME=jdk7: - mvn + zinc - mvn sans zinc - sbt Verified that in all cases, jdk8-specific library calls fail to compile. Author: Marcelo Vanzin <[email protected]> Closes #13272 from vanzin/SPARK-15451. (cherry picked from commit 57adb77e6b0208dc6d29488a674b6ad78702262f) Signed-off-by: Marcelo Vanzin <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ef8d16ec Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/ef8d16ec Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/ef8d16ec Branch: refs/heads/branch-2.0 Commit: ef8d16ec4d566c004a1fd526dfe65b5df7a27b62 Parents: df4f871 Author: Marcelo Vanzin <[email protected]> Authored: Tue May 31 16:54:34 2016 -0700 Committer: Marcelo Vanzin <[email protected]> Committed: Tue May 31 16:54:51 2016 -0700 ---------------------------------------------------------------------- build/mvn | 6 +++++- external/java8-tests/pom.xml | 1 + pom.xml | 39 +++++++++++++++++++++++++++++++++++++++ project/SparkBuild.scala | 16 ++++++++++++++-- 4 files changed, 59 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/ef8d16ec/build/mvn ---------------------------------------------------------------------- diff --git a/build/mvn b/build/mvn index a78b93a..c3ab62d 100755 --- a/build/mvn +++ b/build/mvn @@ -141,9 +141,13 @@ cd "${_CALLING_DIR}" # Now that zinc is ensured to be installed, check its status and, if its # not running or just installed, start it if [ -n "${ZINC_INSTALL_FLAG}" -o -z "`"${ZINC_BIN}" -status -port ${ZINC_PORT}`" ]; then + ZINC_JAVA_HOME= + if [ -n "$JAVA_7_HOME" ]; then + ZINC_JAVA_HOME="env JAVA_HOME=$JAVA_7_HOME" + fi export ZINC_OPTS=${ZINC_OPTS:-"$_COMPILE_JVM_OPTS"} "${ZINC_BIN}" -shutdown -port ${ZINC_PORT} - "${ZINC_BIN}" -start -port ${ZINC_PORT} \ + $ZINC_JAVA_HOME "${ZINC_BIN}" -start -port ${ZINC_PORT} \ -scala-compiler "${SCALA_COMPILER}" \ -scala-library "${SCALA_LIBRARY}" &>/dev/null fi http://git-wip-us.apache.org/repos/asf/spark/blob/ef8d16ec/external/java8-tests/pom.xml ---------------------------------------------------------------------- diff --git a/external/java8-tests/pom.xml b/external/java8-tests/pom.xml index 60e3ff6..74a3ee1 100644 --- a/external/java8-tests/pom.xml +++ b/external/java8-tests/pom.xml @@ -106,6 +106,7 @@ <groupId>net.alchim31.maven</groupId> <artifactId>scala-maven-plugin</artifactId> <configuration> + <useZincServer>${useZincForJdk8}</useZincServer> <javacArgs> <javacArg>-source</javacArg> <javacArg>1.8</javacArg> http://git-wip-us.apache.org/repos/asf/spark/blob/ef8d16ec/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index fff5560..60c8c8d 100644 --- a/pom.xml +++ b/pom.xml @@ -184,6 +184,9 @@ <test.java.home>${java.home}</test.java.home> <test.exclude.tags></test.exclude.tags> + <!-- When using different JDKs for the build, we can't use Zinc for the jdk8 part. --> + <useZincForJdk8>true</useZincForJdk8> + <!-- Package to use when relocating shaded classes. --> <spark.shade.packageName>org.spark_project</spark.shade.packageName> @@ -2577,6 +2580,42 @@ </profile> <profile> + <id>java7</id> + <activation> + <property><name>env.JAVA_7_HOME</name></property> + </activation> + <properties> + <useZincForJdk8>false</useZincForJdk8> + </properties> + <build> + <pluginManagement> + <plugins> + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-compiler-plugin</artifactId> + <configuration> + <compilerArgs combine.children="append"> + <arg>-bootclasspath</arg> + <arg>${env.JAVA_7_HOME}/jre/lib/rt.jar</arg> + </compilerArgs> + </configuration> + </plugin> + <plugin> + <groupId>net.alchim31.maven</groupId> + <artifactId>scala-maven-plugin</artifactId> + <configuration> + <args combine.children="append"> + <arg>-javabootclasspath</arg> + <arg>${env.JAVA_7_HOME}/jre/lib/rt.jar</arg> + </args> + </configuration> + </plugin> + </plugins> + </pluginManagement> + </build> + </profile> + + <profile> <id>scala-2.11</id> <activation> <property><name>!scala-2.10</name></property> http://git-wip-us.apache.org/repos/asf/spark/blob/ef8d16ec/project/SparkBuild.scala ---------------------------------------------------------------------- diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index f08ca70..744f57c 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -277,12 +277,24 @@ object SparkBuild extends PomBuild { // additional discussion and explanation. javacOptions in (Compile, compile) ++= Seq( "-target", javacJVMVersion.value - ), + ) ++ sys.env.get("JAVA_7_HOME").toSeq.flatMap { jdk7 => + if (javacJVMVersion.value == "1.7") { + Seq("-bootclasspath", s"$jdk7/jre/lib/rt.jar") + } else { + Nil + } + }, scalacOptions in Compile ++= Seq( s"-target:jvm-${scalacJVMVersion.value}", "-sourcepath", (baseDirectory in ThisBuild).value.getAbsolutePath // Required for relative source links in scaladoc - ), + ) ++ sys.env.get("JAVA_7_HOME").toSeq.flatMap { jdk7 => + if (javacJVMVersion.value == "1.7") { + Seq("-javabootclasspath", s"$jdk7/jre/lib/rt.jar") + } else { + Nil + } + }, // Implements -Xfatal-warnings, ignoring deprecation warnings. // Code snippet taken from https://issues.scala-lang.org/browse/SI-8410. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
