This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new d99e00c34130 [SPARK-54049][BUILD][4.0] Shade com.google.thirdparty
package to fix Guava class conflicts in spark 4.0
d99e00c34130 is described below
commit d99e00c341303e6a0aefe882a69e276316a8f783
Author: vinodkc <[email protected]>
AuthorDate: Tue Nov 4 15:29:41 2025 +0900
[SPARK-54049][BUILD][4.0] Shade com.google.thirdparty package to fix Guava
class conflicts in spark 4.0
Backport https://github.com/apache/spark/pull/52767 to Spark 4.0 branch
We upgraded Guava from 14.0.1 to 30+ in spark 4.0 . Guava 33.4.0 used in
Spark 4 consists of two main packages:
- `com.google.common`
- `com.google.thirdparty`
Prior to this PR, only the `com.google.common` package was shaded into the
spark-network-common jar, while classes under `com.google.thirdparty` remained
unshaded in the spark-network-common jar. This partial shading causes
classloading conflicts and runtime errors when a downstream project depends on
both Spark and its own version of Guava.
Eg: calls to guava class `com.google.common.net.InternetDomainName` fails
with the following error:
```
Caused by: java.lang.NoSuchFieldError: EXACT
at
com.google.common.net.InternetDomainName.findSuffixOfType(InternetDomainName.java:226)
at
com.google.common.net.InternetDomainName.publicSuffixIndex(InternetDomainName.java:185)
at
com.google.common.net.InternetDomainName.hasPublicSuffix(InternetDomainName.java:400)
at com.eadx.Domain$.printDomainInfo(Domain.scala:16)
at com.eadx.TestApp$.main(TestApp.scala:16)
```
**Root Cause**:
`com.google.common.net.InternetDomainName` uses classes from
`com.google.thirdparty.publicsuffix`.
The classloader resolves `com.google.common.net.InternetDomainName` from
the downstream Guava jar, while
`com.google.thirdparty.publicsuffix.PublicSuffixPatterns` is loaded from Spark
4.x Guava classes, leading to binary incompatibility.
Example diagnostic:
```
InternetDomainName → guava-32.0.0-jre.jar
(target/.../guava-32.0.0-jre.jar)
PublicSuffixPatterns → spark-network-common_2.13-4.0.0.jar
(target/.../spark-network-common_2.13-4.0.0.jar)
```
### What changes were proposed in this pull request?
This PR ensures package `com.google.thirdparty` is also shaded and
isolated under the sparkproject namespace in Spark, preventing downstream class
conflicts and runtime errors.
### Why are the changes needed?
These changes are necessary to prevent runtime errors and class conflicts
for downstream projects that depend on both Spark and Guava by restoring proper
isolation of shaded Guava classes in spark
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
No new test cases added; used existing UT and IT.
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #52869 from vinodkc/br_shade_guava_thirdparty_4.0.
Authored-by: vinodkc <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
pom.xml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/pom.xml b/pom.xml
index 01ce893bb188..a77cda52d3dd 100644
--- a/pom.xml
+++ b/pom.xml
@@ -3150,6 +3150,10 @@
<pattern>com.google.common</pattern>
<shadedPattern>${spark.shade.packageName}.guava</shadedPattern>
</relocation>
+ <relocation>
+ <pattern>com.google.thirdparty</pattern>
+
<shadedPattern>${spark.shade.packageName}.guava.thirdparty</shadedPattern>
+ </relocation>
<relocation>
<pattern>org.dmg.pmml</pattern>
<shadedPattern>${spark.shade.packageName}.dmg.pmml</shadedPattern>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
