This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 5d38830 [SPARK-53781] Exclude Spark's transitive dependencies
consistently across modules
5d38830 is described below
commit 5d38830b8d97549e08d89ed1090b2682bc4d8227
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Oct 1 23:23:46 2025 -0700
[SPARK-53781] Exclude Spark's transitive dependencies consistently across
modules
### What changes were proposed in this pull request?
This PR aims to exclude `Spark`'s transitive dependencies consistently
across modules.
### Why are the changes needed?
Currently `spark-operator` and `spark-submission-worker` modules have
heterogeneous exclusion rules against Apache Spark. To be complete, we need to
be consistent.
https://github.com/apache/spark-kubernetes-operator/blob/2366928e114e663d77ef0c890b59fab72a26ebc1/spark-operator/build.gradle#L41-L51
https://github.com/apache/spark-kubernetes-operator/blob/2366928e114e663d77ef0c890b59fab72a26ebc1/spark-submission-worker/build.gradle#L23-L33
This PR proposes **the following sorted list** which is a union of the
existing exclusions for both modules.
```
exclude group: "com.github.luben"
exclude group: "commons-collections", module: "commons-collections'
exclude group: "io.fabric8"
exclude group: "io.netty", module: "netty-tcnative-boringssl-static"
exclude group: "io.netty", module: "netty-tcnative-classes"
exclude group: "org.apache.logging.log4j"
exclude group: "org.fusesource.leveldbjni"
exclude group: "org.lz4"
exclude group: "org.rocksdb"
exclude group: "org.slf4j"
exclude group: "org.xerial.snappy"
```
### Does this PR introduce _any_ user-facing change?
No behavior change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #361 from dongjoon-hyun/SPARK-53781.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
spark-operator/build.gradle | 1 +
spark-submission-worker/build.gradle | 10 ++++++++--
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/spark-operator/build.gradle b/spark-operator/build.gradle
index 4b7e364..a893257 100644
--- a/spark-operator/build.gradle
+++ b/spark-operator/build.gradle
@@ -40,6 +40,7 @@ dependencies {
implementation(libs.metrics.jvm)
compileOnly(libs.spark.core) {
exclude group: "com.github.luben"
+ exclude group: "commons-collections", module: "commons-collections"
exclude group: "io.netty", module: "netty-tcnative-boringssl-static"
exclude group: "io.netty", module: "netty-tcnative-classes"
exclude group: "org.apache.logging.log4j"
diff --git a/spark-submission-worker/build.gradle
b/spark-submission-worker/build.gradle
index fa9bd60..69d0b9e 100644
--- a/spark-submission-worker/build.gradle
+++ b/spark-submission-worker/build.gradle
@@ -19,17 +19,23 @@
dependencies {
implementation project(":spark-operator-api")
implementation(libs.kubernetes.client)
+ implementation(libs.log4j.api)
+ implementation(libs.log4j.core)
+ implementation(libs.log4j.slf4j2.impl)
+ implementation(libs.slf4j.api)
implementation(libs.spark.kubernetes) {
- exclude group: "io.fabric8"
exclude group: "com.github.luben"
+ exclude group: "commons-collections", module: "commons-collections"
+ exclude group: "io.fabric8"
exclude group: "io.netty", module: "netty-tcnative-boringssl-static"
exclude group: "io.netty", module: "netty-tcnative-classes"
+ exclude group: "org.apache.logging.log4j"
exclude group: "org.fusesource.leveldbjni"
exclude group: "org.lz4"
exclude group: "org.rocksdb"
+ exclude group: "org.slf4j"
exclude group: "org.xerial.snappy"
- exclude group: 'commons-collections', module: 'commons-collections'
}
compileOnly(libs.lombok)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
