In a similar vein to what Levi is proposing, how about exposing a plugin interface in prometheus that external projects can implement? We can provide a UI to select the plugins that the user needs to bake into the prometheus binary before downloading
Something like - https://caddyserver.com/download On Thursday, 1 July 2021 at 11:28:45 UTC+5:30 [email protected] wrote: > Just as a point of personal experience on use of AD in production. Having > tried in a few places: > > Seasonality of human activity rarely lines up well with the data slicing > needed for AD, You get *a lot* of false positives. It's almost always > basically unusable for pagable alerting. > > You need a lot of traffic. Doing stats on infrequent requests, or on > systems that he state thier own traffic spikes (like batch jobs that cause > traffic of any size close to typical usage), just doesn't work well. > > A lot of simpler AD models have a loopback problem where you normal > traffic looks anomalous sone time after an actual event , this causes > another source of annoying false positive that result in people ignoring > alerts. > > There are AD models now that avoid some of the false positive issues, but > I think it remains true that they would be useless without sufficient > traffic volumes (last paper I read was from twitter I think). > > Personally I think any features along these lines will be good fodder for > blog posts, but unlikely to end up useful for real world monitoring. (I > speak as an author of related posts). > > > https://medium.com/qubit-engineering/using-seasonality-in-prometheus-alerting-d90e68337a4c > > On Wed, 30 Jun 2021, 22:24 Bjoern Rabenstein, <[email protected]> wrote: > >> On 25.06.21 01:27, Shandong Dong wrote: >> > Ok, I will try the PR first. Can I know what‘s the concern of >> "Personally, >> > I'm still not sure if that's a sustainable approach. "? >> >> We had a handful of requests in the past to add specific advanced >> statistics functions. In one case, a function was actually added, see >> >> https://prometheus.io/docs/prometheus/latest/querying/functions/#holt_winters >> >> The problem with the latter is that it was actually not the variety of >> Holt-Winters that most people wanted. A lot of misunderstanding >> happened because of that. My impression is (but I might be proven >> wrong) is that this is a rarely used PromQL function. But now we have >> to support it at least until the next major release. >> >> That latter problem will be avoided by feature flags. But if we now >> each of the five to ten persons that requested new functions will add >> on average two to three new functions, we end up with about 20 new >> functions, all with the same potential of being misunderstand. Many >> might be overlapping, so any new function needs to be reviewed for >> overlap with existing ones. Even if they are all behind feature flags, >> they will require a lot of code with potential interaction with >> existing code and with each other, so there is some maintenance >> overhead. >> >> Eventually, reviewing and acceptance of even more functions behind >> feature flags will slow down. So we are back at square one. And the >> multitude of experimental functions will make it harder for users to >> find the right one to try out. Which in turn will make it harder to >> identify the actually generally useful functions and "graduate" >> them. Realistically, there will be small groups of users liking >> subsets of functions, but rarely functions that aither everyone needs >> and likes or nobody. >> >> It feels a bit like the attempt to create a Python interpreter for >> data science that doesn't understand modules and instead tries to have >> all required functions built-in. That's hardly a reasonably >> approach. And that's why my personal idea is that Prometheus either >> has to keep stating that it is only meant for basic mathematical >> operations on metrics, or it has to provide some kind of "scripting >> interface" to allow custom mathematical "libraries" for users with >> special requirements. >> >> -- >> Björn Rabenstein >> [PGP-ID] 0x851C3DA17D748D03 >> [email] [email protected] >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/20210630212438.GO11559%40jahnn >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/09212950-0caf-4fe5-ba39-7ff62ed7aef2n%40googlegroups.com.
