npawar commented on PR #9011: URL: https://github.com/apache/pinot/pull/9011#issuecomment-1182587553
> > > Helix provides a history of all tasks queued. I believe there is a way to configure the retention time of the history before deleting it. Does that not work for you? cc: @jackjlli , @snleee > > > Also, can we do better than a Utils class with static methods? (Hard to unit test). > > > > > > This PR is for Generator side errors I believe, not for the task errors > > Hmm.. so we are using zk like a log? Why not just add a metric and set alerts on it? > > You can also pipe logs into log processors. > > We don't want to pollute ZK with information that can be got from other means. I think it's okay, given we're limiting it to just 5 recent tasks. Most minion related errors and confusion is seen by users when getting started. Given the popularity of trying to use the minion tasks for ingestion and rollups, we're seeing an influx of questions in the community slack, about minion related debugging. Most of the time, the users are asking if there's an API to quickly see the scheduling side and task execution side errors. Plus during the getting started phase, metrics reporting backend, alerting or log processors isn't usually setup. This will be very helpful for such users to quickly debug their issues during the getting started process. This will be used in the Minion UI tab that's come up in the Pinot UI now: https://github.com/apache/pinot/pull/8978 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
