This is an automated email from the ASF dual-hosted git repository.
zhengruifeng pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new d6fefc1fef09 [SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being
silently skipped by unittest
d6fefc1fef09 is described below
commit d6fefc1fef0983323be5d1fc706ada0b8934c419
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Thu May 14 15:56:05 2026 +0800
[SPARK-56861][PS][TESTS] Fix `ArrowInterfaceTests` being silently skipped
by unittest
### What changes were proposed in this pull request?
Fix the base class of `ArrowInterfaceTests` in
`python/pyspark/pandas/tests/test_arrow_interface.py` so the test is actually
discovered by `unittest`.
The class was declared as:
```python
class ArrowInterfaceTests(ArrowInterfaceTestsMixin, SQLTestUtils):
pass
```
`SQLTestUtils` is a plain mixin that does not extend `unittest.TestCase`,
so this class itself is not a `TestCase` subclass and `unittest`'s discovery
(`TestLoader.loadTestsFromModule`) silently skips it.
Verified with `loader.loadTestsFromModule(...).countTestCases()`:
| | tests collected | `issubclass(TestCase)` |
| --- | --- | --- |
| Before fix | 0 | `False` |
| After fix | 3 | `True` |
The sibling Spark Connect parity test `ArrowInterfaceParityTests` in
`tests/connect/test_parity_arrow_interface.py` extends `ReusedConnectTestCase`
(which does chain to `TestCase`) and so was unaffected, masking the issue.
Fix: switch the base to `PandasOnSparkTestCase`, matching the other
`pyspark.pandas.tests.test_*` classes (e.g. `NamespaceTests`, `UtilsTests`,
`InternalFrameTests`, `SQLTests`). `PandasOnSparkTestCase` chains through
`ReusedSQLTestCase -> ReusedPySparkTestCase -> PySparkBaseTestCase ->
unittest.TestCase`.
### Why are the changes needed?
`test_spark_arrow_c_streamer_arrow_consumer` (added in SPARK-55124) has not
actually been running in CI since it was introduced; it is a latent gap in
coverage of the pandas-on-Spark Arrow C-stream interface.
### Does this PR introduce _any_ user-facing change?
No. Test-only change.
### How was this patch tested?
- Discovery check (above): `unittest.TestLoader.loadTestsFromModule` now
collects the test.
- Ran the test locally:
```
$ python -m unittest -v pyspark.pandas.tests.test_arrow_interface
test_assert_classic_mode (...ArrowInterfaceTests) ... ok
test_spark_arrow_c_streamer_arrow_consumer (...ArrowInterfaceTests) ... ok
test_assert_classic_mode (...PandasOnSparkTestCase) ... ok
----------------------------------------------------------------------
Ran 3 tests in 10.683s
OK
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude opus-4-7
Closes #55870 from zhengruifeng/SPARK-arrow-interface-tests.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Ruifeng Zheng <[email protected]>
(cherry picked from commit 9ab8e43a940ebe6adb9cf32336b59c6aff105eff)
Signed-off-by: Ruifeng Zheng <[email protected]>
---
python/pyspark/pandas/tests/test_arrow_interface.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/pandas/tests/test_arrow_interface.py
b/python/pyspark/pandas/tests/test_arrow_interface.py
index d38462dbe9e1..304c38c756c4 100644
--- a/python/pyspark/pandas/tests/test_arrow_interface.py
+++ b/python/pyspark/pandas/tests/test_arrow_interface.py
@@ -21,7 +21,7 @@ from pyspark.testing.utils import (
pyarrow_requirement_message,
)
from pyspark import pandas as ps
-from pyspark.testing.sqlutils import SQLTestUtils
+from pyspark.testing.pandasutils import PandasOnSparkTestCase
import pandas as pd
@@ -60,7 +60,7 @@ class ArrowInterfaceTestsMixin:
self.assertEqual(result, expected)
-class ArrowInterfaceTests(ArrowInterfaceTestsMixin, SQLTestUtils):
+class ArrowInterfaceTests(ArrowInterfaceTestsMixin, PandasOnSparkTestCase):
pass
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
