andygrove opened a new pull request, #1999: URL: https://github.com/apache/datafusion-ballista/pull/1999
# Which issue does this PR close? Closes #1998. # Rationale for this change The executor logs `Execute task : [...]` at the start of a task and `Finished task : [...]` at the end, but neither reports the task's duration. To spot a slow or skewed task from the logs today you have to correlate the two lines by timestamp or inspect the task-status reports. Per-task duration is the most useful number when scanning executor logs for long-pole stages, so this adds it directly to the finished-task line. # What changes are included in this PR? Capture an `Instant` immediately before `execute_query_stage` and format its `elapsed()` into the `Finished task` log message, at both task-dispatch sites: - `ballista/executor/src/executor_server.rs` — the push-based path - `ballista/executor/src/execution_loop.rs` — the pull-based path The message becomes `Finished task : [<task>] in <duration>` (e.g. `in 2.31s`). A monotonic `Instant` is used rather than differencing the existing wall-clock timestamps so the duration is unaffected by clock adjustments. # Are there any user-facing changes? Only the executor log line changes; the `Finished task` message now includes the elapsed execution time. No API or behavior changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
