andygrove opened a new pull request, #2492: URL: https://github.com/apache/datafusion-ballista/pull/2492
# Which issue does this PR close? No separate issue. Split out of #2416 at review request, so it can be reviewed and backported on its own. # Rationale for this change Under `PushStaged`, when `TaskManager::launch_multi_task` cannot prepare a stage's task definition it logged the error and dropped the tasks. Those tasks were never sent to an executor, so no status ever came back for them. The job stayed `Running` for the life of the scheduler, everything waiting on it (a `DistributedQueryExec`, for example) blocked with no timeout, and the vcores bound to the tasks were never refunded. #2477 and #2016 already handle the neighbouring case where an executor rejects a job's tasks: the job is failed and its slots are freed. This routes unpreparable tasks through the same path. # What changes are included in this PR? - `launch_multi_task` adds the job of any stage whose task definition cannot be prepared to the set of failed jobs it returns, alongside the jobs the executor rejected. `launch_tasks` then refunds those tasks' slots and fails the job, as it already does for rejections. - The failure message posted for these jobs now covers both causes. - A test that binds two jobs, makes one job's tasks unpreparable, and checks that only that job is reported failed and that exactly its slots are freed. It fails without the fix. # Are there any user-facing changes? A job whose tasks cannot be prepared now fails with an error instead of hanging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
