This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new c07c3339cfe6 [SPARK-54049][BUILD] Shade com.google.thirdparty package
to fix Guava class conflicts in spark 4.0
c07c3339cfe6 is described below
commit c07c3339cfe62e9a0f04b869383824636c91b789
Author: vinodkc <[email protected]>
AuthorDate: Tue Nov 4 11:54:10 2025 +0900
[SPARK-54049][BUILD] Shade com.google.thirdparty package to fix Guava class
conflicts in spark 4.0
We upgraded Guava from 14.0.1 to 30+ in spark 4.0 . Guava 33.4.0 used in
Spark 4 consists of two main packages:
- `com.google.common`
- `com.google.thirdparty`
Prior to this PR, only the `com.google.common` package was shaded into the
spark-network-common jar, while classes under `com.google.thirdparty` remained
unshaded in the spark-network-common jar. This partial shading causes
classloading conflicts and runtime errors when a downstream project depends on
both Spark and its own version of Guava.
Eg: calls to guava class `com.google.common.net.InternetDomainName` fails
with the following error:
```
Caused by: java.lang.NoSuchFieldError: EXACT
at
com.google.common.net.InternetDomainName.findSuffixOfType(InternetDomainName.java:226)
at
com.google.common.net.InternetDomainName.publicSuffixIndex(InternetDomainName.java:185)
at
com.google.common.net.InternetDomainName.hasPublicSuffix(InternetDomainName.java:400)
at com.eadx.Domain$.printDomainInfo(Domain.scala:16)
at com.eadx.TestApp$.main(TestApp.scala:16)
```
**Root Cause**:
`com.google.common.net.InternetDomainName` uses classes from
`com.google.thirdparty.publicsuffix`.
The classloader resolves `com.google.common.net.InternetDomainName` from
the downstream Guava jar, while
`com.google.thirdparty.publicsuffix.PublicSuffixPatterns` is loaded from Spark
4.x Guava classes, leading to binary incompatibility.
Example diagnostic:
```
InternetDomainName → guava-32.0.0-jre.jar
(target/.../guava-32.0.0-jre.jar)
PublicSuffixPatterns → spark-network-common_2.13-4.0.0.jar
(target/.../spark-network-common_2.13-4.0.0.jar)
```
### What changes were proposed in this pull request?
This PR ensures package `com.google.thirdparty` is also shaded and
isolated under the sparkproject namespace in Spark, preventing downstream class
conflicts and runtime errors.
### Why are the changes needed?
These changes are necessary to prevent runtime errors and class conflicts
for downstream projects that depend on both Spark and Guava by restoring proper
isolation of shaded Guava classes in spark
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
No new test cases added; used existing UT and IT.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #52767 from vinodkc/br_shade_guava_thirdparty.
Authored-by: vinodkc <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit c8cb3e7506c4f36e6abb931237774383e981425d)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
pom.xml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/pom.xml b/pom.xml
index b1c250a34c72..107def928cbb 100644
--- a/pom.xml
+++ b/pom.xml
@@ -3148,6 +3148,10 @@
<pattern>com.google.common</pattern>
<shadedPattern>${spark.shade.packageName}.guava</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>com.google.thirdparty</pattern>
+
<shadedPattern>${spark.shade.packageName}.guava.thirdparty</shadedPattern>
+ </relocation>
<relocation>
<pattern>org.dmg.pmml</pattern>
<shadedPattern>${spark.shade.packageName}.dmg.pmml</shadedPattern>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
