abhishekbafna opened a new pull request, #17190: URL: https://github.com/apache/pinot/pull/17190
This PR adds comprehensive timing metrics for minion task subtasks, enabling better monitoring and observability of task execution. ## Functionality This PR introduces the ability to track and monitor timing information for individual subtasks within minion tasks: - **Subtask waiting time**: Tracks how long subtasks wait before execution begins. This is measured from the job start time until the subtask starts running, providing visibility into queue delays and resource contention. - **Subtask running time**: Tracks how long subtasks take to execute once they start running. This is measured from when the subtask begins execution, providing visibility into actual processing time. Both metrics are collected per subtask and aggregated at the table level, allowing operators to: - Monitor performance of individual subtasks - Identify bottlenecks in task execution - Analyze table-specific task performance - Debug issues related to task timing and delays The PR also consolidates task status retrieval to return organized task status information, providing a unified view of task counts and timing data together. This enables more comprehensive monitoring by combining task state information with timing metrics in a single data structure. ## Benefits - **Improved observability**: Operators can now monitor how long subtasks wait before execution and how long they take to run - **Better debugging**: Timing information helps identify bottlenecks and performance issues in task execution - **Enhanced monitoring**: Metrics are emitted per table via ControllerTimer, allowing for table-specific performance analysis and alerting - **Real-time insights**: Metrics are collected and emitted continuously, providing up-to-date visibility into task execution patterns - **Unified task status view**: Task counts and timing information are now available together, making it easier to correlate task states with performance metrics -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
