suddendust commented on issue #13277: URL: https://github.com/apache/pinot/issues/13277#issuecomment-2175392268
>But in other scenarios (1 and 3), metrics stops getting published rather than a clear differentiation that CONSUMING segment is ONLINE but not consuming @tibrewalpratik17 Yes, that metric is not reported when the consumer loop breaks, so you'll observe gaps in the metric. Can you check if this works for you? ``` (sum by (table, topic, partition) (pinot_server_llcPartitionConsuming_Value{} or last_over_time(pinot_server_llcPartitionConsuming_Value[2d]) * 0) == 0) AND on(table) (pinot_controller_tableDisabled_Value{} == 0 AND pinot_controller_tableConsumptionPaused_Value{} == 0) ``` This takes care of the gaps, cases when tables are disabled/paused. Just make sure you give a long enough eval period for the alert, as there are gaps in actual cases as well (these gaps are pretty small, less than 2m). Try having an eval period of 30m. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org