This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 8887d53117ee [SPARK-50395][SQL] Fix malformed URI syntax in Windows
8887d53117ee is described below
commit 8887d53117eeca7bb085cd4f9ebc1eae11e54748
Author: Paddy Xu <[email protected]>
AuthorDate: Mon Nov 25 08:49:16 2024 +0900
[SPARK-50395][SQL] Fix malformed URI syntax in Windows
### What changes were proposed in this pull request?
This PR fixes an issue that the Artifact Manager is using a malformed URI
string for Class Dir in windows. The issue is caused by using a
platform-specific `File.separator` instead of `/`: Windows's file separator is
`\`, which results in a wrong URI string:
```
java.net.URISyntaxException: Illegal character in path at index 88:
spark://xxxx:57839/artifacts\bd3e1ffe-50d2-412c-8fe4-911ae160c251\classes\
```
This failure is captured by the scheduled Windows build on `master`, such
as https://github.com/apache/spark/actions/runs/11958030827.
To fix this issue we just make sure that the separator is always `/` on all
OSes.
### Why are the changes needed?
Fix a compilation failure in Windows.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Tested on my own fork with a modified Workflow that runs on PR:
https://github.com/xupefei/spark/pull/1,
https://github.com/xupefei/spark/actions/runs/11970330735/job/33372836765?pr=1
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #48934 from xupefei/repl-class-uri-windows.
Authored-by: Paddy Xu <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.../main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git
a/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
b/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
index d362c5bef878..0aae5a43ca40 100644
---
a/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
+++
b/sql/core/src/main/scala/org/apache/spark/sql/artifact/ArtifactManager.scala
@@ -66,12 +66,11 @@ class ArtifactManager(session: SparkSession) extends
Logging {
// The base directory/URI where all artifacts are stored for this
`sessionUUID`.
protected[artifact] val (artifactPath, artifactURI): (Path, String) =
(ArtifactUtils.concatenatePaths(artifactRootPath, session.sessionUUID),
- s"$artifactRootURI${File.separator}${session.sessionUUID}")
+ s"$artifactRootURI/${session.sessionUUID}")
// The base directory/URI where all class file artifacts are stored for this
`sessionUUID`.
protected[artifact] val (classDir, replClassURI): (Path, String) =
- (ArtifactUtils.concatenatePaths(artifactPath, "classes"),
- s"$artifactURI${File.separator}classes${File.separator}")
+ (ArtifactUtils.concatenatePaths(artifactPath, "classes"),
s"$artifactURI/classes/")
private lazy val alwaysApplyClassLoader =
session.conf.get(SQLConf.ARTIFACTS_SESSION_ISOLATION_ALWAYS_APPLY_CLASSLOADER.key).toBoolean
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
