phillipleblanc opened a new pull request, #2037:
URL: https://github.com/apache/datafusion-ballista/pull/2037
# Which issue does this PR close?
Closes #2036
# Rationale for this change
Terminal jobs were removed from the scheduler's active cache before their
final state was saved. During that window, status requests could fall back to
shared state and incorrectly report the job as `Running`.
Abort handling had a related failure mode. If terminal-state persistence
failed, `abort_job` returned before dispatching cancellation for running
executor tasks.
This PR keeps terminal jobs visible in the active cache until persistence
finishes. It applies the same ordering to successful, failed, and cancelled
jobs.
# What changes are included in this PR?
- Persist terminal state before removing jobs from the active cache.
- Count only cached `Running` jobs in the running-job metric.
- Dispatch executor task cancellation before abort persistence, so save
failures cannot suppress cancellation.
# Are there any user-facing changes?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
