This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new 630f4dd [SPARK-31713][INFRA] Make test-dependencies.sh detect version
string correctly
630f4dd is described below
commit 630f4dd2b2bb1d33f8c85cc3506a55f0ad70ef7f
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu May 14 19:28:25 2020 -0700
[SPARK-31713][INFRA] Make test-dependencies.sh detect version string
correctly
### What changes were proposed in this pull request?
This PR makes `test-dependencies.sh` detect the version string correctly by
ignoring all the other lines.
### Why are the changes needed?
Currently, all SBT jobs are broken like the following.
-
https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-branch-3.0-test-sbt-hadoop-3.2-hive-2.3/476/console
```
[error] running
/home/jenkins/workspace/spark-branch-3.0-test-sbt-hadoop-3.2-hive-2.3/dev/test-dependencies.sh
; received return code 1
Build step 'Execute shell' marked build as failure
```
The reason is that the script detects the old version like `Falling back to
archive.apache.org to download Maven 3.1.0-SNAPSHOT` when `build/mvn` did
fallback.
Specifically, in the script, `OLD_VERSION` became `Falling back to
archive.apache.org to download Maven 3.1.0-SNAPSHOT` instead of
`3.1.0-SNAPSHOT` if build/mvn did fallback. Then, `pom.xml` file is corrupted
like the following at the end and the exit code become `1` instead of `0`. It
causes Jenkins jobs fails
```
- <version>3.1.0-SNAPSHOT</version>
+ <version>Falling</version>
```
**NO FALLBACK**
```
$ build/mvn -q -Dexec.executable="echo" -Dexec.args='${project.version}'
--non-recursive org.codehaus.mojo:exec-maven-plugin:1.6.0:exec
Using `mvn` from path:
/Users/dongjoon/APACHE/spark-merge/build/apache-maven-3.6.3/bin/mvn
3.1.0-SNAPSHOT
```
**FALLBACK**
```
$ build/mvn -q -Dexec.executable="echo" -Dexec.args='${project.version}'
--non-recursive org.codehaus.mojo:exec-maven-plugin:1.6.0:exec
Falling back to archive.apache.org to download Maven
Using `mvn` from path:
/Users/dongjoon/APACHE/spark-merge/build/apache-maven-3.6.3/bin/mvn
3.1.0-SNAPSHOT
```
**In the script**
```
$ echo $(build/mvn -q -Dexec.executable="echo"
-Dexec.args='${project.version}' --non-recursive
org.codehaus.mojo:exec-maven-plugin:1.6.0:exec)
Using `mvn` from path:
/Users/dongjoon/APACHE/spark-merge/build/apache-maven-3.6.3/bin/mvn
Falling back to archive.apache.org to download Maven 3.1.0-SNAPSHOT
```
This PR will prevent irrelevant logs like `Falling back to
archive.apache.org to download Maven`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the PR Builder.
Closes #28532 from dongjoon-hyun/SPARK-31713.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit cd5fbcf9a0151f10553f67bcaa22b8122b3cf263)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/test-dependencies.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dev/test-dependencies.sh b/dev/test-dependencies.sh
index 6b4fa71..58e9112 100755
--- a/dev/test-dependencies.sh
+++ b/dev/test-dependencies.sh
@@ -56,7 +56,7 @@ OLD_VERSION=$($MVN -q \
-Dexec.executable="echo" \
-Dexec.args='${project.version}' \
--non-recursive \
- org.codehaus.mojo:exec-maven-plugin:1.6.0:exec)
+ org.codehaus.mojo:exec-maven-plugin:1.6.0:exec | grep -E
'[0-9]+\.[0-9]+\.[0-9]+')
if [ $? != 0 ]; then
echo -e "Error while getting version string from Maven:\n$OLD_VERSION"
exit 1
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
