This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 282e7d33d673 [SPARK-53862][DSTREAM][TESTS] Fix `CheckpointSuite.'get
correct sparkdriver.[host|port] from checkpoint'` test flakiness
282e7d33d673 is described below
commit 282e7d33d673b3df837f4524b754dfeff750090c
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Oct 9 19:14:34 2025 -0700
[SPARK-53862][DSTREAM][TESTS] Fix `CheckpointSuite.'get correct
sparkdriver.[host|port] from checkpoint'` test flakiness
### What changes were proposed in this pull request?
This PR aims to fix `CheckpointSuite` flakiness.
### Why are the changes needed?
Currently, master branch is flaky.
- https://github.com/apache/spark/actions/runs/18373591172/job/52342350296
```
[info] CheckpointSuite:
[info] - non-existent checkpoint dir (1 millisecond)
[info] - basic rdd checkpoints + dstream graph checkpoint recovery (6
seconds, 787 milliseconds)
[info] - recovery of conf through checkpoints (114 milliseconds)
[info] - get correct spark.driver.[host|port] from checkpoint *** FAILED
*** (76 milliseconds)
[info] "[10000]" did not equal "[9999]" (CheckpointSuite.scala:403)
[info] Analysis:
[info] "[10000]" -> "[9999]"
[info] org.scalatest.exceptions.TestFailedException:
[info] at
org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
[info] at
org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
[info] at
org.scalatest.Assertions$.newAssertionFailedException(Assertions.scala:1231)
[info] at
org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:1295)
[info] at
org.apache.spark.streaming.CheckpointSuite.$anonfun$new$20(CheckpointSuite.scala:403)
```
### Does this PR introduce _any_ user-facing change?
No because this is a test case change.
### How was this patch tested?
Pass the CIs.
I also manually repeat multiple times locally.
```
$ build/sbt "streaming/testOnly *.CheckpointSuite -- -z correct"
[info] CheckpointSuite:
[info] - get correct spark.driver.[host|port] from checkpoint (617
milliseconds)
[info] Run completed in 1 second, 262 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 20 s, completed Oct 9, 2025, 7:06:29 PM
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #52565 from dongjoon-hyun/SPARK-53862.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala | 1 +
1 file changed, 1 insertion(+)
diff --git
a/streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala
index 8a32f07537ff..0ebbcd492044 100644
--- a/streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala
+++ b/streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala
@@ -386,6 +386,7 @@ class CheckpointSuite extends TestSuiteBase with
LocalStreamingContext with DStr
val cp = new Checkpoint(ssc, Time(1000))
ssc.stop()
+ Thread.sleep(100)
// Serialize/deserialize to simulate write to storage and reading it back
val newCp = Utils.deserialize[Checkpoint](Utils.serialize(cp))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
