This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 2fc65e1c98ed [SPARK-54820][PYTHON][4.0][4.1] Make pandas_on_spark_type
compatible with numpy 2.4.0
2fc65e1c98ed is described below
commit 2fc65e1c98ed53641f5204215b840e33463df987
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Thu Dec 25 13:23:27 2025 +0800
[SPARK-54820][PYTHON][4.0][4.1] Make pandas_on_spark_type compatible with
numpy 2.4.0
backport https://github.com/apache/spark/pull/53580 to 4.0 and 4.1,
to restore
https://github.com/apache/spark/actions/runs/20493084511/job/58888601618
Closes #53604 from zhengruifeng/bp_np_24.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit eec9f8fd4c7c08330b19f971df891969b9604697)
Signed-off-by: Ruifeng Zheng <[email protected]>
---
python/pyspark/pandas/typedef/typehints.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/pyspark/pandas/typedef/typehints.py
b/python/pyspark/pandas/typedef/typehints.py
index 48545d124b2d..a4ed9f996fe4 100644
--- a/python/pyspark/pandas/typedef/typehints.py
+++ b/python/pyspark/pandas/typedef/typehints.py
@@ -342,7 +342,7 @@ def pandas_on_spark_type(tpe: Union[str, type, Dtype]) ->
Tuple[Dtype, types.Dat
try:
dtype = pandas_dtype(tpe)
spark_type = as_spark_type(dtype)
- except TypeError:
+ except (TypeError, ValueError):
spark_type = as_spark_type(tpe)
dtype = spark_type_to_pandas_dtype(spark_type)
return dtype, spark_type
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
