This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new a5fc89cbcab2 [SPARK-54127][BUILD] Fix sbt inconsistent shading package
a5fc89cbcab2 is described below
commit a5fc89cbcab2bd1b17bf2ffbb94e1764844edd5c
Author: Cheng Pan <[email protected]>
AuthorDate: Sun Nov 2 20:27:29 2025 -0800
[SPARK-54127][BUILD] Fix sbt inconsistent shading package
### What changes were proposed in this pull request?
Fix sbt inconsistent shading package for `spark-connect` module
### Why are the changes needed?
Take 4.1.0-preview3 as the example, which is built by Maven.
```
$ jar tf jars/spark-connect_2.13-4.1.0-preview3.jar | grep 'connect/grpc' |
head
org/sparkproject/connect/grpc/
org/sparkproject/connect/grpc/inprocess/
org/sparkproject/connect/grpc/inprocess/InProcessTransport$InProcessStream$InProcessServerStream.class
org/sparkproject/connect/grpc/inprocess/InProcessTransport$2.class
org/sparkproject/connect/grpc/inprocess/InProcessTransport$InProcessStream$InProcessClientStream.class
org/sparkproject/connect/grpc/inprocess/InProcessTransport$SingleMessageProducer.class
org/sparkproject/connect/grpc/inprocess/InProcessTransport$6.class
org/sparkproject/connect/grpc/inprocess/InProcessTransport$1.class
org/sparkproject/connect/grpc/inprocess/InProcessSocketAddress.class
org/sparkproject/connect/grpc/inprocess/InProcessTransport$InProcessStream.class
```
the produced jar built by SBT has a different relocated package for grpc
classes
```
$ jar tf jars/spark-connect_2.13-4.1.0-SNAPSHOT.jar | grep 'connect/grpc' |
head
org/sparkproject/connect/grpc/
org/sparkproject/connect/grpc/io/
org/sparkproject/connect/grpc/io/grpc/
org/sparkproject/connect/grpc/io/grpc/Attributes$1.class
org/sparkproject/connect/grpc/io/grpc/Attributes$Builder.class
org/sparkproject/connect/grpc/io/grpc/Attributes$Key.class
org/sparkproject/connect/grpc/io/grpc/Attributes.class
org/sparkproject/connect/grpc/io/grpc/BinaryLog.class
org/sparkproject/connect/grpc/io/grpc/BindableService.class
org/sparkproject/connect/grpc/io/grpc/CallCredentials$MetadataApplier.class
```
### Does this PR introduce _any_ user-facing change?
No, so far I don't see such inconsistent causes real issue.
### How was this patch tested?
Verified the output of `dev/make-distribution.sh --sbt-enabled`
```
jar tf jars/spark-connect_2.13-4.1.0-SNAPSHOT.jar | grep 'connect/grpc' |
head
org/sparkproject/connect/grpc/
org/sparkproject/connect/grpc/Attributes$1.class
org/sparkproject/connect/grpc/Attributes$Builder.class
org/sparkproject/connect/grpc/Attributes$Key.class
org/sparkproject/connect/grpc/Attributes.class
org/sparkproject/connect/grpc/BinaryLog.class
org/sparkproject/connect/grpc/BindableService.class
org/sparkproject/connect/grpc/CallCredentials$MetadataApplier.class
org/sparkproject/connect/grpc/CallCredentials$RequestInfo.class
org/sparkproject/connect/grpc/CallCredentials.class
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52825 from pan3793/SPARK-54127.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 3d92bdf6ac1628dc72d525f8e16ba0790ee9f3ed)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
project/SparkBuild.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index d852b4155bea..1d8de063133e 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -808,7 +808,7 @@ object SparkConnect {
},
(assembly / assemblyShadeRules) := Seq(
- ShadeRule.rename("io.grpc.**" ->
"org.sparkproject.connect.grpc.@0").inAll,
+ ShadeRule.rename("io.grpc.**" ->
"org.sparkproject.connect.grpc.@1").inAll,
ShadeRule.rename("com.google.common.**" ->
"org.sparkproject.connect.guava.@1").inAll,
ShadeRule.rename("com.google.thirdparty.**" ->
"org.sparkproject.connect.guava.@1").inAll,
ShadeRule.rename("com.google.protobuf.**" ->
"org.sparkproject.connect.protobuf.@1").inAll,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
