This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new d37aca54de0b [SPARK-56592][K8S] Skip upgrading setuptools in K8s
PySpark Dockerfile
d37aca54de0b is described below
commit d37aca54de0bddd9054a5f1d9f1e2d860f9b7dcd
Author: Cheng Pan <[email protected]>
AuthorDate: Thu Apr 23 08:39:31 2026 -0700
[SPARK-56592][K8S] Skip upgrading setuptools in K8s PySpark Dockerfile
This is an alternative to SPARK-56583
(https://github.com/apache/spark/pull/55492), which adds
`--break-system-packages` to `pip3 install --no-cache-dir --upgrade setuptools`
to fix the
```
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
...
note: If you believe this is a mistake, please contact your Python
installation or OS distribution provider. You can override this, at the risk of
breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
```
actually, similar to `pip`, `python3-setuptools` is installed already too,
we can simply remove the upgrade
```
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libpython3-stdlib libpython3.12-minimal libpython3.12-stdlib
libreadline8t64 libsqlite3-0 media-types netbase python3-minimal
python3-pkg-resources python3-setuptools python3-wheel python3.12
python3.12-minimal
readline-common
...
```
A better alternative to SPARK-56583, eliminate the
`--break-system-packages` usage.
No.
Pass GHA.
No.
Closes #55504 from pan3793/skip-setuptools.
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 72d6ea06bdcccd037f11ce5f89a5ca3c3716d4b0)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../docker/src/main/dockerfiles/spark/bindings/python/Dockerfile | 1 -
1 file changed, 1 deletion(-)
diff --git
a/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile
b/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile
index 640247dd7408..ab85d57762b2 100644
---
a/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile
+++
b/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile
@@ -31,7 +31,6 @@ USER 0
RUN mkdir ${SPARK_HOME}/python
RUN apt-get update && \
apt install -y python3 python3-pip && \
- pip3 install --break-system-packages --upgrade setuptools && \
# Removed the .cache to save space
rm -rf /root/.cache && rm -rf /var/cache/apt/* && rm -rf
/var/lib/apt/lists/*
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
