Repository: spark Updated Branches: refs/heads/master 921fa40b1 -> 147c02082
[SPARK-15818][BUILD] Upgrade to Hadoop 2.7.2 ## What changes were proposed in this pull request? Updating the Hadoop version from 2.7.0 to 2.7.2 if we use the Hadoop-2.7 build profile ## How was this patch tested? (Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests) Existing tests (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) I'd like us to use Hadoop 2.7.2 owing to the Hadoop release notes stating Hadoop 2.7.0 is not ready for production use https://hadoop.apache.org/docs/r2.7.0/ states "Apache Hadoop 2.7.0 is a minor release in the 2.x.y release line, building upon the previous stable release 2.6.0. This release is not yet ready for production use. Production users should use 2.7.1 release and beyond." Hadoop 2.7.1 release notes: "Apache Hadoop 2.7.1 is a minor release in the 2.x.y release line, building upon the previous release 2.7.0. This is the next stable release after Apache Hadoop 2.6.x." And then Hadoop 2.7.2 release notes: "Apache Hadoop 2.7.2 is a minor release in the 2.x.y release line, building upon the previous stable release 2.7.1." I've tested this is OK with Intel hardware and IBM Java 8 so let's test it with OpenJDK, ideally this will be pushed to branch-2.0 and master. Author: Adam Roberts <[email protected]> Closes #13556 from a-roberts/patch-2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/147c0208 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/147c0208 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/147c0208 Branch: refs/heads/master Commit: 147c020823080c60b495f7950629d8134bf895db Parents: 921fa40 Author: Adam Roberts <[email protected]> Authored: Thu Jun 9 10:34:01 2016 +0100 Committer: Sean Owen <[email protected]> Committed: Thu Jun 9 10:34:01 2016 +0100 ---------------------------------------------------------------------- dev/deps/spark-deps-hadoop-2.4 | 30 +++++++++++++++--------------- dev/deps/spark-deps-hadoop-2.6 | 30 +++++++++++++++--------------- dev/deps/spark-deps-hadoop-2.7 | 30 +++++++++++++++--------------- pom.xml | 6 +++--- 4 files changed, 48 insertions(+), 48 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/147c0208/dev/deps/spark-deps-hadoop-2.4 ---------------------------------------------------------------------- diff --git a/dev/deps/spark-deps-hadoop-2.4 b/dev/deps/spark-deps-hadoop-2.4 index f0491ec..501bf58 100644 --- a/dev/deps/spark-deps-hadoop-2.4 +++ b/dev/deps/spark-deps-hadoop-2.4 @@ -53,21 +53,21 @@ eigenbase-properties-1.1.5.jar guava-14.0.1.jar guice-3.0.jar guice-servlet-3.0.jar -hadoop-annotations-2.4.0.jar -hadoop-auth-2.4.0.jar -hadoop-client-2.4.0.jar -hadoop-common-2.4.0.jar -hadoop-hdfs-2.4.0.jar -hadoop-mapreduce-client-app-2.4.0.jar -hadoop-mapreduce-client-common-2.4.0.jar -hadoop-mapreduce-client-core-2.4.0.jar -hadoop-mapreduce-client-jobclient-2.4.0.jar -hadoop-mapreduce-client-shuffle-2.4.0.jar -hadoop-yarn-api-2.4.0.jar -hadoop-yarn-client-2.4.0.jar -hadoop-yarn-common-2.4.0.jar -hadoop-yarn-server-common-2.4.0.jar -hadoop-yarn-server-web-proxy-2.4.0.jar +hadoop-annotations-2.4.1.jar +hadoop-auth-2.4.1.jar +hadoop-client-2.4.1.jar +hadoop-common-2.4.1.jar +hadoop-hdfs-2.4.1.jar +hadoop-mapreduce-client-app-2.4.1.jar +hadoop-mapreduce-client-common-2.4.1.jar +hadoop-mapreduce-client-core-2.4.1.jar +hadoop-mapreduce-client-jobclient-2.4.1.jar +hadoop-mapreduce-client-shuffle-2.4.1.jar +hadoop-yarn-api-2.4.1.jar +hadoop-yarn-client-2.4.1.jar +hadoop-yarn-common-2.4.1.jar +hadoop-yarn-server-common-2.4.1.jar +hadoop-yarn-server-web-proxy-2.4.1.jar hk2-api-2.4.0-b34.jar hk2-locator-2.4.0-b34.jar hk2-utils-2.4.0-b34.jar http://git-wip-us.apache.org/repos/asf/spark/blob/147c0208/dev/deps/spark-deps-hadoop-2.6 ---------------------------------------------------------------------- diff --git a/dev/deps/spark-deps-hadoop-2.6 b/dev/deps/spark-deps-hadoop-2.6 index b3dced6..b915727 100644 --- a/dev/deps/spark-deps-hadoop-2.6 +++ b/dev/deps/spark-deps-hadoop-2.6 @@ -58,21 +58,21 @@ gson-2.2.4.jar guava-14.0.1.jar guice-3.0.jar guice-servlet-3.0.jar -hadoop-annotations-2.6.0.jar -hadoop-auth-2.6.0.jar -hadoop-client-2.6.0.jar -hadoop-common-2.6.0.jar -hadoop-hdfs-2.6.0.jar -hadoop-mapreduce-client-app-2.6.0.jar -hadoop-mapreduce-client-common-2.6.0.jar -hadoop-mapreduce-client-core-2.6.0.jar -hadoop-mapreduce-client-jobclient-2.6.0.jar -hadoop-mapreduce-client-shuffle-2.6.0.jar -hadoop-yarn-api-2.6.0.jar -hadoop-yarn-client-2.6.0.jar -hadoop-yarn-common-2.6.0.jar -hadoop-yarn-server-common-2.6.0.jar -hadoop-yarn-server-web-proxy-2.6.0.jar +hadoop-annotations-2.6.4.jar +hadoop-auth-2.6.4.jar +hadoop-client-2.6.4.jar +hadoop-common-2.6.4.jar +hadoop-hdfs-2.6.4.jar +hadoop-mapreduce-client-app-2.6.4.jar +hadoop-mapreduce-client-common-2.6.4.jar +hadoop-mapreduce-client-core-2.6.4.jar +hadoop-mapreduce-client-jobclient-2.6.4.jar +hadoop-mapreduce-client-shuffle-2.6.4.jar +hadoop-yarn-api-2.6.4.jar +hadoop-yarn-client-2.6.4.jar +hadoop-yarn-common-2.6.4.jar +hadoop-yarn-server-common-2.6.4.jar +hadoop-yarn-server-web-proxy-2.6.4.jar hk2-api-2.4.0-b34.jar hk2-locator-2.4.0-b34.jar hk2-utils-2.4.0-b34.jar http://git-wip-us.apache.org/repos/asf/spark/blob/147c0208/dev/deps/spark-deps-hadoop-2.7 ---------------------------------------------------------------------- diff --git a/dev/deps/spark-deps-hadoop-2.7 b/dev/deps/spark-deps-hadoop-2.7 index 16f60f2..f752eaa 100644 --- a/dev/deps/spark-deps-hadoop-2.7 +++ b/dev/deps/spark-deps-hadoop-2.7 @@ -58,21 +58,21 @@ gson-2.2.4.jar guava-14.0.1.jar guice-3.0.jar guice-servlet-3.0.jar -hadoop-annotations-2.7.0.jar -hadoop-auth-2.7.0.jar -hadoop-client-2.7.0.jar -hadoop-common-2.7.0.jar -hadoop-hdfs-2.7.0.jar -hadoop-mapreduce-client-app-2.7.0.jar -hadoop-mapreduce-client-common-2.7.0.jar -hadoop-mapreduce-client-core-2.7.0.jar -hadoop-mapreduce-client-jobclient-2.7.0.jar -hadoop-mapreduce-client-shuffle-2.7.0.jar -hadoop-yarn-api-2.7.0.jar -hadoop-yarn-client-2.7.0.jar -hadoop-yarn-common-2.7.0.jar -hadoop-yarn-server-common-2.7.0.jar -hadoop-yarn-server-web-proxy-2.7.0.jar +hadoop-annotations-2.7.2.jar +hadoop-auth-2.7.2.jar +hadoop-client-2.7.2.jar +hadoop-common-2.7.2.jar +hadoop-hdfs-2.7.2.jar +hadoop-mapreduce-client-app-2.7.2.jar +hadoop-mapreduce-client-common-2.7.2.jar +hadoop-mapreduce-client-core-2.7.2.jar +hadoop-mapreduce-client-jobclient-2.7.2.jar +hadoop-mapreduce-client-shuffle-2.7.2.jar +hadoop-yarn-api-2.7.2.jar +hadoop-yarn-client-2.7.2.jar +hadoop-yarn-common-2.7.2.jar +hadoop-yarn-server-common-2.7.2.jar +hadoop-yarn-server-web-proxy-2.7.2.jar hk2-api-2.4.0-b34.jar hk2-locator-2.4.0-b34.jar hk2-utils-2.4.0-b34.jar http://git-wip-us.apache.org/repos/asf/spark/blob/147c0208/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 6c67452..fd40683 100644 --- a/pom.xml +++ b/pom.xml @@ -2484,7 +2484,7 @@ <profile> <id>hadoop-2.4</id> <properties> - <hadoop.version>2.4.0</hadoop.version> + <hadoop.version>2.4.1</hadoop.version> <jets3t.version>0.9.3</jets3t.version> </properties> </profile> @@ -2492,7 +2492,7 @@ <profile> <id>hadoop-2.6</id> <properties> - <hadoop.version>2.6.0</hadoop.version> + <hadoop.version>2.6.4</hadoop.version> <jets3t.version>0.9.3</jets3t.version> <zookeeper.version>3.4.6</zookeeper.version> <curator.version>2.6.0</curator.version> @@ -2502,7 +2502,7 @@ <profile> <id>hadoop-2.7</id> <properties> - <hadoop.version>2.7.0</hadoop.version> + <hadoop.version>2.7.2</hadoop.version> <jets3t.version>0.9.3</jets3t.version> <zookeeper.version>3.4.6</zookeeper.version> <curator.version>2.6.0</curator.version> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
