This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new a8f817d1ad9d [SPARK-54745][PYTHON] Fix PySpark import error caused by
missing UnixStreamServer on Windows
a8f817d1ad9d is described below
commit a8f817d1ad9df6bb2eac7a3b6645e55a4379a6c6
Author: Kristin Cowalcijk <[email protected]>
AuthorDate: Sun Dec 21 14:45:28 2025 +0900
[SPARK-54745][PYTHON] Fix PySpark import error caused by missing
UnixStreamServer on Windows
### What changes were proposed in this pull request?
This PR fixes an error caused by `socketserver.UnixStreamServer` not being
available on Windows. We define a fallback `AccumulatorUnixServer` to raise an
exception on construction and inform the user to disable
`spark.python.unix.domain.socket.enabled`.
### Why are the changes needed?
`import pyspark` fails with the following message on Windows since PySpark
4.1.0:
```
sedona\spark\__init__.py:19: in <module>
import pyspark
.venv\Lib\site-packages\pyspark\__init__.py:71: in <module>
from pyspark.accumulators import Accumulator, AccumulatorParam
.venv\Lib\site-packages\pyspark\accumulators.py:324: in <module>
class AccumulatorUnixServer(socketserver.UnixStreamServer):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E AttributeError: module 'socketserver' has no attribute
'UnixStreamServer'
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually test this on Windows 11
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #53546 from Kontinuation/fix-uds-windows-compat.
Authored-by: Kristin Cowalcijk <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/accumulators.py | 45 ++++++++++++++++++++++++++++--------------
1 file changed, 30 insertions(+), 15 deletions(-)
diff --git a/python/pyspark/accumulators.py b/python/pyspark/accumulators.py
index 32de0fcf5fae..4465bde9bbfb 100644
--- a/python/pyspark/accumulators.py
+++ b/python/pyspark/accumulators.py
@@ -345,21 +345,36 @@ class AccumulatorTCPServer(socketserver.TCPServer):
self.server_close()
-class AccumulatorUnixServer(socketserver.UnixStreamServer):
- server_shutdown = False
-
- def __init__(
- self, socket_path: str, RequestHandlerClass:
Type[socketserver.BaseRequestHandler]
- ):
- super().__init__(socket_path, RequestHandlerClass)
- self.auth_token = None
-
- def shutdown(self) -> None:
- self.server_shutdown = True
- super().shutdown()
- self.server_close()
- if os.path.exists(self.server_address): # type: ignore[arg-type]
- os.remove(self.server_address) # type: ignore[arg-type]
+# socketserver.UnixStreamServer is not available on Windows yet
+# (https://github.com/python/cpython/issues/77589).
+if hasattr(socketserver, "UnixStreamServer"):
+
+ class AccumulatorUnixServer(socketserver.UnixStreamServer):
+ server_shutdown = False
+
+ def __init__(
+ self, socket_path: str, RequestHandlerClass:
Type[socketserver.BaseRequestHandler]
+ ):
+ super().__init__(socket_path, RequestHandlerClass)
+ self.auth_token = None
+
+ def shutdown(self) -> None:
+ self.server_shutdown = True
+ super().shutdown()
+ self.server_close()
+ if os.path.exists(self.server_address): # type: ignore[arg-type]
+ os.remove(self.server_address) # type: ignore[arg-type]
+
+else:
+
+ class AccumulatorUnixServer(socketserver.TCPServer): # type:
ignore[no-redef]
+ def __init__(
+ self, socket_path: str, RequestHandlerClass:
Type[socketserver.BaseRequestHandler]
+ ):
+ raise NotImplementedError(
+ "Unix Domain Sockets are not supported on this platform. "
+ "Please disable it by setting
spark.python.unix.domain.socket.enabled to false."
+ )
def _start_update_server(
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
