This is an automated email from the ASF dual-hosted git repository.
yangjie01 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ced87308a23e [SPARK-53077][CORE][TESTS] Add an env to control the
execution of the `SparkBloomFilterSuite`
ced87308a23e is described below
commit ced87308a23e15f368a3164bf4132580a172cf60
Author: yangjie01 <[email protected]>
AuthorDate: Mon Aug 4 22:13:35 2025 +0800
[SPARK-53077][CORE][TESTS] Add an env to control the execution of the
`SparkBloomFilterSuite`
### What changes were proposed in this pull request?
This pr adds an environment variable named
`SPARK_TEST_SPARK_BLOOM_FILTER_SUITE_ENABLED` to control whether the test case
`SparkBloomFilterSuite` is executed. It also ensures that this test case is
only run for validation in the daily tests specified in `build_non_ansi.yml`.
### Why are the changes needed?
The `SparkBloomFilterSuite` requires periodic validation, but due to its
excessively long execution time (over 10 minutes), it is not suitable for
execution in the Change Pipeline.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manual verification:
- maven
```
build/mvn package --pl common/sketch
[INFO] Running org.apache.spark.util.sketch.SparkBloomFilterSuite
[WARNING] Tests run: 2, Failures: 0, Errors: 0, Skipped: 2, Time elapsed:
0.001 s -- in org.apache.spark.util.sketch.SparkBloomFilterSuite
```
```
SPARK_TEST_SPARK_BLOOM_FILTER_SUITE_ENABLED=true build/mvn package --pl
common/sketch
[INFO] Running org.apache.spark.util.sketch.SparkBloomFilterSuite
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
401.9 s -- in org.apache.spark.util.sketch.SparkBloomFilterSuite
```
- sbt
```
build/sbt clean "sketch/test"
[info] Test run started (JUnit Jupiter)
[info] Test org.apache.spark.util.sketch.SparkBloomFilterSuite ignored:
Environment variable [SPARK_TEST_SPARK_BLOOM_FILTER_SUITE_ENABLED] does not
exist
[info] Test run finished: 0 failed, 0 ignored, 0 total, 0.016s
```
```
SPARK_TEST_SPARK_BLOOM_FILTER_SUITE_ENABLED=true build/sbt clean
"sketch/test"
[info] Test run started (JUnit Jupiter)
[info] Test
org.apache.spark.util.sketch.SparkBloomFilterSuite#testAccuracyRandomDistribution(long,
double, int, org.junit.jupiter.api.TestInfo):#1 started
[info] Test
org.apache.spark.util.sketch.SparkBloomFilterSuite#testAccuracyEvenOdd(long,
double, int, org.junit.jupiter.api.TestInfo):#1 started
[info] Test run finished: 0 failed, 0 ignored, 2 total, 359.776s
```
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #51806 from LuciferYang/SPARK-53077.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: yangjie01 <[email protected]>
---
.github/workflows/build_non_ansi.yml | 1 +
.../test/java/org/apache/spark/util/sketch/SparkBloomFilterSuite.java | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/build_non_ansi.yml
b/.github/workflows/build_non_ansi.yml
index 547a227e61d7..debdaf4f8709 100644
--- a/.github/workflows/build_non_ansi.yml
+++ b/.github/workflows/build_non_ansi.yml
@@ -40,6 +40,7 @@ jobs:
"PYSPARK_IMAGE_TO_TEST": "python-311",
"PYTHON_TO_TEST": "python3.11",
"SPARK_ANSI_SQL_MODE": "false",
+ "SPARK_TEST_SPARK_BLOOM_FILTER_SUITE_ENABLED": "true"
}
jobs: >-
{
diff --git
a/common/sketch/src/test/java/org/apache/spark/util/sketch/SparkBloomFilterSuite.java
b/common/sketch/src/test/java/org/apache/spark/util/sketch/SparkBloomFilterSuite.java
index 529ba5419338..a7186853edfc 100644
---
a/common/sketch/src/test/java/org/apache/spark/util/sketch/SparkBloomFilterSuite.java
+++
b/common/sketch/src/test/java/org/apache/spark/util/sketch/SparkBloomFilterSuite.java
@@ -18,6 +18,7 @@
package org.apache.spark.util.sketch;
import org.junit.jupiter.api.*;
+import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
@@ -33,7 +34,8 @@ import java.util.concurrent.atomic.LongAdder;
import java.util.stream.LongStream;
import java.util.stream.Stream;
-@Disabled("TODO(SPARK-53077): Re-enable with a resonable test time.")
+@EnabledIfEnvironmentVariable(
+ named = "SPARK_TEST_SPARK_BLOOM_FILTER_SUITE_ENABLED", matches = "true")
public class SparkBloomFilterSuite {
// the implemented fpp limit is only approximating the hard boundary,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
