This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new b29c8223639b [SPARK-54695][TEST][CORE]
StandaloneDynamicAllocationSuite.syncExecutors should ensure executors have
fully setup
b29c8223639b is described below
commit b29c8223639bb795f57cf0a9a531750a3ed507f1
Author: Yi Wu <[email protected]>
AuthorDate: Mon Dec 15 10:47:14 2025 +0800
[SPARK-54695][TEST][CORE] StandaloneDynamicAllocationSuite.syncExecutors
should ensure executors have fully setup
### What changes were proposed in this pull request?
This PR fixes `StandaloneDynamicAllocationSuite.syncExecutors()` to ensure
executors have fully setup before it returns.
### Why are the changes needed?
After SPARK-29287, executor gets fully setup in two steps: 1.
`RegisterExecutor`, 2. `LaunchedExecutor`. And the executor is only ready for
use (`freeCores` > 0) when the driver has processed the `LaunchedExecutor`.
`StandaloneDynamicAllocationSuite.syncExecutors()` was unintentionally broken
by SPARK-29287, leading to unexpected access failures upon executor's
`freeCores`.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Covered by existing tests (improve the statabilty)
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #53457 from Ngone51/fix-syncExecutors.
Authored-by: Yi Wu <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 45a2604df52ad97758c3081f0cf6e2b13b668983)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala
b/core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala
index 5ecc551c16b8..487a90f157a9 100644
---
a/core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala
+++
b/core/src/test/scala/org/apache/spark/deploy/StandaloneDynamicAllocationSuite.scala
@@ -632,6 +632,9 @@ class StandaloneDynamicAllocationSuite
Map.empty, ResourceProfile.DEFAULT_RESOURCE_PROFILE_ID)
backend.driverEndpoint.askSync[Boolean](message)
backend.driverEndpoint.send(LaunchedExecutor(id))
+ eventually(timeout(10.seconds), interval(100.millis)) {
+ assert(backend.getExecutorAvailableCpus(id).exists(_ > 0))
+ }
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
