This is an automated email from the ASF dual-hosted git repository.
pdallig pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zeppelin.git
The following commit(s) were added to refs/heads/master by this push:
new 60d565ce47 [ZEPPELIN-5836] Remove scala-2.10 because it has no JDK 11
Support (#4494)
60d565ce47 is described below
commit 60d565ce472b2bf9ed3b5386de3f239349225eb5
Author: Philipp Dallig <philipp.dal...@gmail.com>
AuthorDate: Tue Oct 25 08:40:32 2022 +0200
[ZEPPELIN-5836] Remove scala-2.10 because it has no JDK 11 Support (#4494)
* Scala adjustments with jdk11
* Update some interpreter to support jdk11
---
.github/workflows/core.yml | 4 +-
dev/change_scala_version.sh | 2 +-
docs/setup/basics/how_to_build.md | 2 +-
docs/setup/deployment/flink_and_spark_cluster.md | 10 ++---
flink/flink-scala-parent/pom.xml | 8 +---
jdbc/pom.xml | 6 +--
livy/pom.xml | 2 +-
pom.xml | 33 ++++-----------
spark/interpreter/pom.xml | 14 ------
zeppelin-display/pom.xml | 54 +++++++++++++-----------
10 files changed, 52 insertions(+), 83 deletions(-)
diff --git a/.github/workflows/core.yml b/.github/workflows/core.yml
index a31ba5a73d..66eb1fbf12 100644
--- a/.github/workflows/core.yml
+++ b/.github/workflows/core.yml
@@ -108,7 +108,7 @@ jobs:
restore-keys: |
${{ runner.os }}-zeppelin-
- name: install environment
- run: ./mvnw install -DskipTests -DskipRat -am -pl
.,zeppelin-interpreter,zeppelin-interpreter-shaded,${INTERPRETERS} -Pscala-2.10
-B
+ run: ./mvnw install -DskipTests -DskipRat -am -pl
.,zeppelin-interpreter,zeppelin-interpreter-shaded,${INTERPRETERS} -Pscala-2.11
-B
- name: Setup conda environment with python 3.7 and R
uses: conda-incubator/setup-miniconda@v2
with:
@@ -121,7 +121,7 @@ jobs:
auto-activate-base: false
use-mamba: true
- name: verify interpreter
- run: ./mvnw verify -DskipRat -pl ${INTERPRETERS} -Pscala-2.10 -B
+ run: ./mvnw verify -DskipRat -pl ${INTERPRETERS} -Pscala-2.11 -B
# test interpreter modules for jupyter, python, rlang
interpreter-test-jupyter-python-rlang:
diff --git a/dev/change_scala_version.sh b/dev/change_scala_version.sh
index 0ccfe7e263..5713945077 100755
--- a/dev/change_scala_version.sh
+++ b/dev/change_scala_version.sh
@@ -69,7 +69,7 @@ sed_i
'1,/<scala\.binary\.version>[0-9]*\.[0-9]*</s/<scala\.binary\.version>[0-9
# update <scala.version> in parent POM
# This is to make variables in leaf pom to be substituted to real value when
flattened-pom is created.
-# maven-flatten plugin doesn't take properties defined under profile even if
scala-2.11/scala-2.10 is activated via -Pscala-2.11/-Pscala-2.10,
+# maven-flatten plugin doesn't take properties defined under profile even if
scala-2.11/scala-2.12 is activated via -Pscala-2.11/-Pscala-2.12,
# and use default defined properties to create flatten pom.
sed_i
'1,/<scala\.version>[0-9]*\.[0-9]*\.[0-9]*</s/<scala\.version>[0-9]*\.[0-9]*\.[0-9]*</<scala.version>'${SCALA_LIB_VERSION}'</'
\
"${BASEDIR}/pom.xml"
diff --git a/docs/setup/basics/how_to_build.md
b/docs/setup/basics/how_to_build.md
index df5620af29..d31dcb8f5a 100644
--- a/docs/setup/basics/how_to_build.md
+++ b/docs/setup/basics/how_to_build.md
@@ -87,8 +87,8 @@ To be noticed, this scala profile affect the modules (e.g.
cassandra) that use s
Set scala version (default 2.10). Available profiles are
```
--Pscala-2.10
-Pscala-2.11
+-Pscala-2.12
```
#### Spark Interpreter
diff --git a/docs/setup/deployment/flink_and_spark_cluster.md
b/docs/setup/deployment/flink_and_spark_cluster.md
index 90fde4464c..d395ccab67 100644
--- a/docs/setup/deployment/flink_and_spark_cluster.md
+++ b/docs/setup/deployment/flink_and_spark_cluster.md
@@ -92,26 +92,26 @@ cd zeppelin
Package Zeppelin.
```bash
-./mvnw clean package -DskipTests -Pspark-1.6 -Dflink.version=1.1.3 -Pscala-2.10
+./mvnw clean package -DskipTests -Pspark-3.2 -Dflink.version=1.1.3 -Pscala-2.11
```
`-DskipTests` skips build tests- you're not developing (yet), so you don't
need to do tests, the clone version *should* build.
-`-Pspark-1.6` tells maven to build a Zeppelin with Spark 1.6. This is
important because Zeppelin has its own Spark interpreter and the versions must
be the same.
+`-Pspark-3.2` tells maven to build a Zeppelin with Spark 3.2. This is
important because Zeppelin has its own Spark interpreter and the versions must
be the same.
`-Dflink.version=1.1.3` tells maven specifically to build Zeppelin with Flink
version 1.1.3.
--`-Pscala-2.10` tells maven to build with Scala v2.10.
+-`-Pscala-2.11` tells maven to build with Scala v2.11.
-**Note:** You can build against any version of Spark that has a Zeppelin build
profile available. The key is to make sure you check out the matching version
of Spark to build. At the time of this writing, Spark 1.6 was the most recent
Spark version available.
+**Note:** You can build against any version of Spark that has a Zeppelin build
profile available. The key is to make sure you check out the matching version
of Spark to build. At the time of this writing, Spark 3.2 was the most recent
Spark version available.
**Note:** On build failures. Having installed Zeppelin close to 30 times now,
I will tell you that sometimes the build fails for seemingly no reason.
As long as you didn't edit any code, it is unlikely the build is failing
because of something you did. What does tend to happen, is some dependency that
maven is trying to download is unreachable. If your build fails on this step
here are some tips:
- Don't get discouraged.
- Scroll up and read through the logs. There will be clues there.
-- Retry (that is, run the `./mvnw clean package -DskipTests -Pspark-1.6` again)
+- Retry (that is, run the `./mvnw clean package -DskipTests -Pspark-3.2` again)
- If there were clues that a dependency couldn't be downloaded wait a few
hours or even days and retry again. Open source software when compiling is
trying to download all of the dependencies it needs, if a server is off-line
there is nothing you can do but wait for it to come back.
- Make sure you followed all of the steps carefully.
- Ask the community to help you. Go
[here](http://zeppelin.apache.org/community.html) and join the user mailing
list. People are there to help you. Make sure to copy and paste the build
output (everything that happened in the console) and include that in your
message.
diff --git a/flink/flink-scala-parent/pom.xml b/flink/flink-scala-parent/pom.xml
index fdf5a1b692..af2464ac10 100644
--- a/flink/flink-scala-parent/pom.xml
+++ b/flink/flink-scala-parent/pom.xml
@@ -29,7 +29,6 @@
<artifactId>flink-scala-parent</artifactId>
<packaging>pom</packaging>
- <version>0.11.0-SNAPSHOT</version>
<name>Zeppelin: Flink Scala Parent</name>
<description>Zeppelin Flink Scala Parent</description>
@@ -37,7 +36,7 @@
<!--library versions-->
<interpreter.name>flink</interpreter.name>
<flink.version>${flink1.12.version}</flink.version>
- <flink.hadoop.version>2.6.5</flink.hadoop.version>
+ <flink.hadoop.version>${hadoop2.7.version}</flink.hadoop.version>
<hive.version>2.3.4</hive.version>
<hiverunner.version>4.0.0</hiverunner.version>
<grpc.version>1.15.0</grpc.version>
@@ -575,7 +574,7 @@
<dependency>
<groupId>com.klarna</groupId>
<artifactId>hiverunner</artifactId>
- <version>4.0.0</version>
+ <version>${hiverunner.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
@@ -1051,7 +1050,6 @@
<hiverunner.version>4.0.0</hiverunner.version>
</properties>
</profile>
-
<profile>
<id>hive1</id>
<properties>
@@ -1067,7 +1065,5 @@
</dependency>
</dependencies>
</profile>
-
</profiles>
-
</project>
diff --git a/jdbc/pom.xml b/jdbc/pom.xml
index 5ec3dc6150..1b3d9b20e5 100644
--- a/jdbc/pom.xml
+++ b/jdbc/pom.xml
@@ -35,10 +35,10 @@
<!--library versions-->
<interpreter.name>jdbc</interpreter.name>
<postgresql.version>42.4.1</postgresql.version>
- <hadoop.version>${hadoop2.7.version}</hadoop.version>
+ <hadoop.version>${hadoop3.1.version}</hadoop.version>
<h2.version>2.0.206</h2.version>
<commons.dbcp2.version>2.0.1</commons.dbcp2.version>
- <hive2.version>2.3.4</hive2.version>
+ <hive3.version>3.1.3</hive3.version>
<!--test library versions-->
<mockrunner.jdbc.version>1.0.8</mockrunner.jdbc.version>
@@ -147,7 +147,7 @@
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-jdbc</artifactId>
- <version>${hive2.version}</version>
+ <version>${hive3.version}</version>
<scope>provided</scope>
<exclusions>
<exclusion>
diff --git a/livy/pom.xml b/livy/pom.xml
index 477b53c863..70743f438b 100644
--- a/livy/pom.xml
+++ b/livy/pom.xml
@@ -40,7 +40,7 @@
<!--test library versions-->
<livy.version>0.7.1-incubating</livy.version>
- <spark.version>2.1.0</spark.version>
+ <spark.version>2.4.8</spark.version>
<hadoop.version>${hadoop2.7.version}</hadoop.version>
</properties>
diff --git a/pom.xml b/pom.xml
index ea0eb18bf5..d3cfe8567e 100644
--- a/pom.xml
+++ b/pom.xml
@@ -32,7 +32,7 @@
<parent>
<groupId>org.apache</groupId>
<artifactId>apache</artifactId>
- <version>17</version>
+ <version>26</version>
</parent>
<licenses>
@@ -97,11 +97,11 @@
<properties>
<!-- language versions -->
+ <maven.compiler.target>1.8</maven.compiler.target>
<java.version>1.8</java.version>
- <scala.2.10.version>2.10.5</scala.2.10.version>
- <scala.version>${scala.2.10.version}</scala.version>
- <scala.binary.version>2.10</scala.binary.version>
- <scala.2.11.version>2.11.8</scala.2.11.version>
+ <scala.version>${scala.2.11.version}</scala.version>
+ <scala.binary.version>2.11</scala.binary.version>
+ <scala.2.11.version>2.11.12</scala.2.11.version>
<scala.2.12.version>2.12.16</scala.2.12.version>
<scalatest.version>3.0.7</scalatest.version>
<scalacheck.version>1.12.5</scalacheck.version>
@@ -188,7 +188,7 @@
<plugin.remote.resource.version>1.7.0</plugin.remote.resource.version>
<plugin.resource.version>3.1.0</plugin.resource.version>
<plugin.s3.upload.version>1.4</plugin.s3.upload.version>
- <plugin.scala.alchim31.version>3.4.6</plugin.scala.alchim31.version>
+ <plugin.scala.alchim31.version>4.6.3</plugin.scala.alchim31.version>
<plugin.scala.tools.version>2.15.2</plugin.scala.tools.version>
<plugin.scalate.version>1.7.1</plugin.scalate.version>
<plugin.scalatest.version>2.0.0</plugin.scalatest.version>
@@ -1800,12 +1800,6 @@
<version>${plugin.scalate.version}</version>
</plugin>
- <plugin>
- <groupId>org.scala-tools</groupId>
- <artifactId>maven-scala-plugin</artifactId>
- <version>${plugin.scala.tools.version}</version>
- </plugin>
-
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
@@ -1835,17 +1829,6 @@
</build>
<profiles>
- <profile>
- <id>scala-2.10</id>
- <activation>
- <activeByDefault>true</activeByDefault>
- </activation>
- <properties>
- <scala.version>${scala.2.10.version}</scala.version>
- <scala.binary.version>2.10</scala.binary.version>
- </properties>
- </profile>
-
<profile>
<id>scala-2.11</id>
<properties>
@@ -1855,10 +1838,10 @@
</profile>
<profile>
<id>scala-2.12</id>
- <properties>
+ <properties>
<scala.version>${scala.2.12.version}</scala.version>
<scala.binary.version>2.12</scala.binary.version>
- </properties>
+ </properties>
</profile>
<profile>
diff --git a/spark/interpreter/pom.xml b/spark/interpreter/pom.xml
index 8f154ba609..65d44bc78f 100644
--- a/spark/interpreter/pom.xml
+++ b/spark/interpreter/pom.xml
@@ -70,20 +70,6 @@
<groupId>org.apache.zeppelin</groupId>
<artifactId>zeppelin-display</artifactId>
<version>${project.version}</version>
- <exclusions>
- <exclusion>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-library</artifactId>
- </exclusion>
- <exclusion>
- <groupId>org.scala-lang</groupId>
- <artifactId>scala-compiler</artifactId>
- </exclusion>
- <exclusion>
- <groupId>org.scala-lang</groupId>
- <artifactId>scalap</artifactId>
- </exclusion>
- </exclusions>
</dependency>
<dependency>
diff --git a/zeppelin-display/pom.xml b/zeppelin-display/pom.xml
index af75b4ca49..751e3a726d 100644
--- a/zeppelin-display/pom.xml
+++ b/zeppelin-display/pom.xml
@@ -33,7 +33,6 @@
<properties>
<!--plugin versions -->
<plugin.failsafe.version>2.16</plugin.failsafe.version>
- <plugin.scala.version>2.15.2</plugin.scala.version>
<plugin.scalatest.version>1.0</plugin.scalatest.version>
</properties>
@@ -89,20 +88,6 @@
</dependency>
</dependencies>
- <profiles>
- <profile>
- <id>scala-2.11</id>
- <dependencies>
- <dependency>
- <groupId>org.scala-lang.modules</groupId>
- <artifactId>scala-xml_${scala.binary.version}</artifactId>
- <version>1.1.0</version>
- <scope>provided</scope>
- </dependency>
- </dependencies>
- </profile>
- </profiles>
-
<build>
<plugins>
<plugin>
@@ -121,30 +106,49 @@
</plugin>
<plugin>
- <groupId>org.scala-tools</groupId>
- <artifactId>maven-scala-plugin</artifactId>
+ <groupId>net.alchim31.maven</groupId>
+ <artifactId>scala-maven-plugin</artifactId>
<executions>
<execution>
- <id>compile</id>
+ <id>eclipse-add-source</id>
<goals>
- <goal>compile</goal>
+ <goal>add-source</goal>
</goals>
- <phase>compile</phase>
</execution>
<execution>
- <id>test-compile</id>
+ <id>scala-compile-first</id>
+ <phase>process-resources</phase>
<goals>
- <goal>testCompile</goal>
+ <goal>compile</goal>
</goals>
- <phase>test-compile</phase>
</execution>
<execution>
- <phase>process-resources</phase>
+ <id>scala-test-compile-first</id>
+ <phase>process-test-resources</phase>
<goals>
- <goal>compile</goal>
+ <goal>testCompile</goal>
</goals>
</execution>
</executions>
+ <configuration>
+ <args>
+ <arg>-unchecked</arg>
+ <arg>-deprecation</arg>
+ <arg>-feature</arg>
+ </args>
+ <jvmArgs>
+ <jvmArg>-Xms1024m</jvmArg>
+ <jvmArg>-Xmx1024m</jvmArg>
+ <jvmArg>-XX:MaxMetaspaceSize=${MaxMetaspace}</jvmArg>
+ </jvmArgs>
+ <javacArgs>
+ <javacArg>-source</javacArg>
+ <javacArg>${java.version}</javacArg>
+ <javacArg>-target</javacArg>
+ <javacArg>${java.version}</javacArg>
+ <javacArg>-Xlint:all,-serial,-path,-options</javacArg>
+ </javacArgs>
+ </configuration>
</plugin>
<plugin>
