This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new d5d0bd83ceb [SPARK-43516][PYTHON][FOLLOWUP] Use pyspark cloudpickle
instead of cloudpickle
d5d0bd83ceb is described below
commit d5d0bd83ceb73620eaa802d1152f46967fd4a7be
Author: Weichen Xu <[email protected]>
AuthorDate: Fri May 26 19:32:39 2023 -0700
[SPARK-43516][PYTHON][FOLLOWUP] Use pyspark cloudpickle instead of
cloudpickle
### What changes were proposed in this pull request?
Use `pyspark.cloudpickle` instead of `cloudpickle`
### Why are the changes needed?
All pyspark code should use pyspark.cloudpickle instead of `cloudpickle`
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
UT.
Closes #41327 from WeichenXu123/use-pyspark-cloudpickle.
Authored-by: Weichen Xu <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
python/pyspark/mlv2/util.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/pyspark/mlv2/util.py b/python/pyspark/mlv2/util.py
index efdc421866e..de2ffb3d7c1 100644
--- a/python/pyspark/mlv2/util.py
+++ b/python/pyspark/mlv2/util.py
@@ -16,10 +16,10 @@
#
import pandas as pd
-import cloudpickle # type: ignore[import]
from collections.abc import Callable, Iterable
from typing import Any, Union
+from pyspark import cloudpickle
from pyspark.sql import DataFrame
from pyspark.sql.functions import col, pandas_udf
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
