This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 9c40c126b799 [SPARK-53946][BUILD] Upgrade SBT to 1.11.7
9c40c126b799 is described below
commit 9c40c126b7996a9c8620f0f67d076b7244967eea
Author: Kousuke Saruta <[email protected]>
AuthorDate: Mon Oct 20 14:53:21 2025 -0700
[SPARK-53946][BUILD] Upgrade SBT to 1.11.7
### What changes were proposed in this pull request?
This PR aims to upgrade SBT to 1.11.7
Release Notes:
* 1.9.4: https://github.com/sbt/sbt/releases/tag/v1.9.4
* 1.9.5: https://github.com/sbt/sbt/releases/tag/v1.9.5
* 1.9.6: https://github.com/sbt/sbt/releases/tag/v1.9.6
* 1.9.7: https://github.com/sbt/sbt/releases/tag/v1.9.7
* 1.9.8: https://github.com/sbt/sbt/releases/tag/v1.9.8
* 1.9.9: https://github.com/sbt/sbt/releases/tag/v1.9.9
* 1.10.0: https://github.com/sbt/sbt/releases/tag/v1.10.0
* 1.10.1: https://github.com/sbt/sbt/releases/tag/v1.10.1
* 1.10.2: https://github.com/sbt/sbt/releases/tag/v1.10.2
* 1.10.3: https://github.com/sbt/sbt/releases/tag/v1.10.3
* 1.10.4: https://github.com/sbt/sbt/releases/tag/v1.10.4
* 1.10.5: https://github.com/sbt/sbt/releases/tag/v1.10.5
* 1.10.6: https://github.com/sbt/sbt/releases/tag/v1.10.6
* 1.10.7: https://github.com/sbt/sbt/releases/tag/v1.10.7
* 1.10.8: https://github.com/sbt/sbt/releases/tag/v1.10.8
* 1.10.9: https://github.com/sbt/sbt/releases/tag/v1.10.9
* 1.10.10: https://github.com/sbt/sbt/releases/tag/v1.10.10
* 1.10.11: https://github.com/sbt/sbt/releases/tag/v1.10.11
* 1.11.0: https://github.com/sbt/sbt/releases/tag/v1.11.0
* 1.11.1: https://github.com/sbt/sbt/releases/tag/v1.11.1
* 1.11.2: https://github.com/sbt/sbt/releases/tag/v1.11.2
* 1.11.3: https://github.com/sbt/sbt/releases/tag/v1.11.3
* 1.11.4: https://github.com/sbt/sbt/releases/tag/v1.11.4
* 1.11.5: https://github.com/sbt/sbt/releases/tag/v1.11.5
* 1.11.6: https://github.com/sbt/sbt/releases/tag/v1.11.6
* 1.11.7: https://github.com/sbt/sbt/releases/tag/v1.11.7
Difference between 1.9.3 and 1.11.7 is as follows.
https://github.com/sbt/sbt/compare/v1.9.3...v1.11.7
To upgrade SBT, this PR also changes SparkBuild.scala to exclude
`slf4j-api` from mllib-local.
Without this change, running ./dev/test-dependencies.sh and ./dev/mima in
this order will result in failure.
```
./dev/test-dependencies.sh
...
./dev/mima
[error]
file:/root/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar: not
found: /root/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar
[error]
[error] at
lmcoursier.internal.shaded.coursier.Artifacts$.$anonfun$fetchArtifacts$9(Artifacts.scala:365)
[error] at
lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$extension$1(Task.scala:14)
[error] at
lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$extension$1$adapted(Task.scala:14)
[error] at
lmcoursier.internal.shaded.coursier.util.Task$.wrap(Task.scala:82)
[error] at
lmcoursier.internal.shaded.coursier.util.Task$.$anonfun$flatMap$2(Task.scala:14)
[error] at
scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
[error] at
scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:51)
[error] at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
[error] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[error] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] at java.base/java.lang.Thread.run(Thread.java:840)
[error] Caused by:
lmcoursier.internal.shaded.coursier.cache.ArtifactError$NotFound: not found:
/root/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar
[error] at
lmcoursier.internal.shaded.coursier.cache.internal.Downloader.$anonfun$checkFileExists$1(Downloader.scala:603)
[error] at
scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
[error] at scala.util.Success.$anonfun$map$1(Try.scala:255)
[error] at scala.util.Success.map(Try.scala:213)
[error] at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
[error] at
scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
[error] at
scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
[error] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[error] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] at java.base/java.lang.Thread.run(Thread.java:840)
[error] (mllib-local / update)
lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts:
Error fetching artifacts:
[error]
file:/root/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar: not
found: /root/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar
```
`slf4j-api-1.7.5` is a transitive dependency of mllib-local.
At runtime, slf4j-api-2.0.17 will be used so `dev/test-dependencies.sh`
downloads `slf4j-api-1.7.5.pom` but does not download `slf4j-api-1.7.5.jar`.
On the other hand, `dev/mima` will fetch `slf4j-api-1.7.5.jar` from `.m2`
even though it's not used at runtime but it's absent in `.m2`.
### Why are the changes needed?
We last upgraded SBT two years ago.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
GA.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52653 from sarutak/upgrade-sbt-1.11.7.
Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
project/SparkBuild.scala | 3 ++-
project/build.properties | 2 +-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index a57aae28a699..23ef5d01808a 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -1099,7 +1099,8 @@ object DependencyOverrides {
lazy val settings = Seq(
dependencyOverrides += "com.google.guava" % "guava" % guavaVersion,
dependencyOverrides += "jline" % "jline" % "2.14.6",
- dependencyOverrides += "org.apache.avro" % "avro" % "1.12.0")
+ dependencyOverrides += "org.apache.avro" % "avro" % "1.12.0",
+ dependencyOverrides += "org.slf4j" % "slf4j-api" % "2.0.17")
}
/**
diff --git a/project/build.properties b/project/build.properties
index 41f6be168792..4e2aefd693b3 100644
--- a/project/build.properties
+++ b/project/build.properties
@@ -14,4 +14,4 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-sbt.version=1.9.3
+sbt.version=1.11.7
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
