This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 16c0ce961844 [SPARK-52594][DOCS] Workaround pandas version string
issue in doc generation
16c0ce961844 is described below
commit 16c0ce961844854bca310d2b01bf843668935ce2
Author: Hyukjin Kwon <[email protected]>
AuthorDate: Sat Jun 28 08:32:48 2025 +0900
[SPARK-52594][DOCS] Workaround pandas version string issue in doc generation
### What changes were proposed in this pull request?
This PR proposes to work around the version string issue in pandas
https://github.com/pandas-dev/pandas/issues/61579 by simply manually removing
the string after `+`.
### Why are the changes needed?
Now the release Spark build fails
https://github.com/apache/spark/actions/runs/15920389407/job/44905725248
```
...
2025-06-27T08:41:09.8092197Z Traceback (most recent call last):
2025-06-27T08:41:09.8092911Z File
"/usr/local/lib/python3.9/dist-packages/sphinx/config.py", line 332, in
eval_config_file
2025-06-27T08:41:09.8093443Z exec(code, namespace)
2025-06-27T08:41:09.8093964Z File
"/opt/spark-rm/output/spark/python/docs/source/conf.py", line 33, in <module>
2025-06-27T08:41:09.8094479Z
generate_supported_api(output_rst_file_path)
2025-06-27T08:41:09.8095154Z File
"/opt/spark-rm/output/spark/python/pyspark/pandas/supported_api_gen.py", line
102, in generate_supported_api
2025-06-27T08:41:09.8096186Z _check_pandas_version()
2025-06-27T08:41:09.8096783Z File
"/opt/spark-rm/output/spark/python/pyspark/pandas/supported_api_gen.py", line
116, in _check_pandas_version
2025-06-27T08:41:09.8097420Z raise ImportError(msg)
2025-06-27T08:41:09.8097843Z ImportError: Warning: pandas 2.3.0 is
required; your version is 2.3.0+4.g1dfc98e16a
...
```
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Will monitor the build
### Was this patch authored or co-authored using generative AI tooling?
No,
Closes #51301 from HyukjinKwon/SPARK-52594.
Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/pandas/supported_api_gen.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/python/pyspark/pandas/supported_api_gen.py
b/python/pyspark/pandas/supported_api_gen.py
index 0272311d7f35..62a3e23816ef 100644
--- a/python/pyspark/pandas/supported_api_gen.py
+++ b/python/pyspark/pandas/supported_api_gen.py
@@ -108,7 +108,9 @@ def _check_pandas_version() -> None:
"""
Check if the installed pandas version matches the expected version.
"""
- if LooseVersion(pd.__version__) != LooseVersion(PANDAS_LATEST_VERSION):
+ # Work around pandas version string issue,
+ # see https://github.com/pandas-dev/pandas/issues/61579.
+ if LooseVersion(pd.__version__.split("+")[0]) !=
LooseVersion(PANDAS_LATEST_VERSION):
msg = (
f"Warning: pandas {PANDAS_LATEST_VERSION} is required; your
version is {pd.__version__}"
)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
