This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a commit to branch branch-3.5
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.5 by this push:
new f250ab2cb7b [SPARK-43965][PYTHON][CONNECT][FOLLOWUP] Include
test_parity_udtf in spark test module
f250ab2cb7b is described below
commit f250ab2cb7b2a7a158403a1b5e3ca4e57e934c94
Author: allisonwang-db <[email protected]>
AuthorDate: Wed Jul 19 12:00:31 2023 +0800
[SPARK-43965][PYTHON][CONNECT][FOLLOWUP] Include test_parity_udtf in spark
test module
### What changes were proposed in this pull request?
This is a follow up PR for SPARK-43965 to include `test_parity_udtf` in
spark test module. It also fixes a test failure.
### Why are the changes needed?
To include a new test case.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests
Closes #42065 from allisonwang-db/spark-43965-follow-up.
Authored-by: allisonwang-db <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit a1eea7f206b92963417ce77ee8047705a7b32942)
Signed-off-by: Ruifeng Zheng <[email protected]>
---
dev/sparktestsupport/modules.py | 1 +
python/pyspark/sql/tests/connect/test_parity_udtf.py | 9 ---------
2 files changed, 1 insertion(+), 9 deletions(-)
diff --git a/dev/sparktestsupport/modules.py b/dev/sparktestsupport/modules.py
index 85ef0c494b8..6382e9a5369 100644
--- a/dev/sparktestsupport/modules.py
+++ b/dev/sparktestsupport/modules.py
@@ -852,6 +852,7 @@ pyspark_connect = Module(
"pyspark.sql.tests.connect.test_parity_column",
"pyspark.sql.tests.connect.test_parity_readwriter",
"pyspark.sql.tests.connect.test_parity_udf",
+ "pyspark.sql.tests.connect.test_parity_udtf",
"pyspark.sql.tests.connect.test_parity_pandas_udf",
"pyspark.sql.tests.connect.test_parity_pandas_map",
"pyspark.sql.tests.connect.test_parity_arrow_map",
diff --git a/python/pyspark/sql/tests/connect/test_parity_udtf.py
b/python/pyspark/sql/tests/connect/test_parity_udtf.py
index f5f37f1f5c0..e18e116e003 100644
--- a/python/pyspark/sql/tests/connect/test_parity_udtf.py
+++ b/python/pyspark/sql/tests/connect/test_parity_udtf.py
@@ -104,15 +104,6 @@ class UDTFParityTests(BaseUDTFTestsMixin,
ReusedConnectTestCase):
with self.assertRaisesRegex(SparkConnectGrpcException, err_msg):
TestUDTF(lit(1)).show()
- def test_udtf_with_empty_yield(self):
- @udtf(returnType="a: int")
- class TestUDTF:
- def eval(self, a: int):
- yield
-
- with self.assertRaisesRegex(SparkConnectGrpcException,
"java.lang.NullPointerException"):
- TestUDTF(lit(1)).collect()
-
class ArrowUDTFParityTests(UDTFArrowTestsMixin, UDTFParityTests):
@classmethod
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
