This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 61c87b283e27 [SPARK-55743][PYTHON][TESTS] Add `python/benchmarks` in
python linter
61c87b283e27 is described below
commit 61c87b283e2787cd7ea2d027953be9abb04fc3cb
Author: Ruifeng Zheng <[email protected]>
AuthorDate: Thu Feb 26 22:18:28 2026 -0800
[SPARK-55743][PYTHON][TESTS] Add `python/benchmarks` in python linter
### What changes were proposed in this pull request?
Add `python/benchmarks` in python linter
### Why are the changes needed?
to make python linter works in `python/benchmarks`
### Does this PR introduce _any_ user-facing change?
no, dev-only
### How was this patch tested?
ci and manually run `dev/reformat-python`
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #54535 from zhengruifeng/reformat_benchmark.
Authored-by: Ruifeng Zheng <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/lint-python | 2 +-
dev/reformat-python | 2 +-
python/benchmarks/bench_arrow.py | 6 ++----
3 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/dev/lint-python b/dev/lint-python
index 1ed042b66dfa..a4ee6752d575 100755
--- a/dev/lint-python
+++ b/dev/lint-python
@@ -330,7 +330,7 @@ function black_test {
fi
echo "starting black test..."
- BLACK_REPORT=$( ($BLACK_BUILD --check python/pyspark dev python/packaging)
2>&1)
+ BLACK_REPORT=$( ($BLACK_BUILD --check python/pyspark dev python/packaging
python/benchmarks) 2>&1)
BLACK_STATUS=$?
if [ "$BLACK_STATUS" -ne 0 ]; then
diff --git a/dev/reformat-python b/dev/reformat-python
index 4bc3a733ec2b..19f0ffa74cc7 100755
--- a/dev/reformat-python
+++ b/dev/reformat-python
@@ -29,4 +29,4 @@ if [ $? -ne 0 ]; then
exit 1
fi
-$BLACK_BUILD python/pyspark dev python/packaging
+$BLACK_BUILD python/pyspark dev python/packaging python/benchmarks
diff --git a/python/benchmarks/bench_arrow.py b/python/benchmarks/bench_arrow.py
index 7ace66f22965..29a95dbcc98b 100644
--- a/python/benchmarks/bench_arrow.py
+++ b/python/benchmarks/bench_arrow.py
@@ -35,9 +35,7 @@ class ArrowToPandasBenchmark:
def setup(self, n_rows, types_mapper):
self.int_array = pa.array(np.random.randint(0, 1000, n_rows))
- self.int_array_with_nulls = pa.array(
- [i if i % 10 != 0 else None for i in range(n_rows)]
- )
+ self.int_array_with_nulls = pa.array([i if i % 10 != 0 else None for i
in range(n_rows)])
self.types_mapper = pd.ArrowDtype if types_mapper == "arrow_dtype"
else None
def time_int_to_pandas(self, n_rows, types_mapper):
@@ -95,7 +93,7 @@ class NullableLongArrowToPandasBenchmark:
self.long_array_with_nulls = pa.array(
[i if i % 10 != 0 else None for i in range(n_rows - 1)] +
[9223372036854775707],
type=pa.int64(),
- )
+ )
# check 3 different ways to convert nullable longs to nullable extension
type
def run_long_with_nulls_to_pandas_ext(self, n_rows, method):
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
