If we decide to make extra scrape metrics optional, the extra scrape metrics should not be a flag, but a per-job option (with a global value) changeable a runtime.
If you think they should be optional, it's better to open an issue in the repository to discuss it. Le ven. 12 nov. 2021, 15:34, [email protected] <[email protected]> a écrit : > I recently stumbled upon a ticket that mentioned adding something behind a > feature flag, then in Prometheus 3.x making that the default and removing > the flag. > This approach might work well for changes in behaviour or default > configuration values, but it doesn't sound like it would work for flags > like ----enable-feature=extra-scrape-metrics and I wonder if those would > always stay under --enable-feature flag. > I guess there are a few options: > - promote it to a dedicated flag > - expose those extra metrics under a different path (/metrics/debug ?) so > user still needs to opt-in, it's just that the mechanism is different > - enable them if /metrics endpoint receives some query params > (/metrics?extra-scrape-metrics=true), just a different version of the above > - always produce extra metrics and document how to drop it if it's too much > > I don't think there's anything wrong with having that under a feature > flag, but I do agree that having excessive number of feature flags means > that each user runs a slightly different version of prometheus, with a > unique set of "features" enabled, which won't improve usability and might > be confusing (which flags should I enable and which I shouldn't?). > Plus I don't think it's unique to this one use case to have some metrics > that are considered "debug" or "optional", so there might be a benefit in > some best practice how to expose such metrics when writing instrumentation > code. > > On Friday, 3 September 2021 at 14:11:37 UTC+1 [email protected] wrote: > >> Having feature flags for extra features that a user needs act on to use >> anyway does feel unnecessary from my POV. I feels more like "I acknowledge >> this is beta" box tick. >> I think it mostly makes sense if one is enabling something that changes >> existing behaviour (and so can break existing use cases), possibly with the >> assumption that the flag will be removed in the next major release and the >> behind-the-flag behaviour would become default. >> >> I do think flags might also make sense for enabling extra metrics that >> can be expensive (like scrape timeouts), mostly because the alternative >> would be to have a release notes line announcing new metrics and advising >> to drop those via relabelling if not used - this only works when user is >> upgrading to that release and if they read notes, for any new install it's >> likely to be missed. >> >> On Friday, 3 September 2021 at 08:44:34 UTC+1 Julien Pivotto wrote: >> >>> Dear developers, >>> >>> TL;DR: I'd like to be able to mark feature as experimental in the >>> documentation without feature flag. >>> >>> >>> >>> >>> I am bringing to your attention a discussion we're having in github >>> issue https://github.com/prometheus/prometheus/pull/9248. That github >>> issue is about having atan2 support in Prometheus as a binary operator. >>> >>> Because it is strange for users to have atan written as foo atan2 bar, >>> instead of atan2(foo, bar), I've asked to mark the feature as >>> experimental. Should we in the future add syntactic sugar or add binary >>> ops to functions, we would be able to do so. >>> >>> I also did not want a feature flag for this. In my vision, and that's >>> how I acted as maintainer, feature flags should be used when: >>> >>> - We change an existing behaviour. >>> => PromQL query range semantics, with @ delimiter >>> => Expanding env variables for external labels >>> - We introduce very risky features, that introduce additional memory / >>> storage requirements. >>> => Remote write receiver >>> => Exemplars >>> => Snapshots on shutdown >>> => Additional scrape metrics (next to up etc) >>> >>> However, I am not a believer of feature-flags driven Prometheus. >>> >>> Feature flags have been introduced in Prometheus 2.25. Since then, we >>> have added a few new features without feature flag: >>> >>> - body_size_limit in scrape configs >>> - setting timeout and interval via relabeling. >>> >>> In general, I think it does not benefit users to launch Prometheus with >>> lots of feature flags. Our users should be able to assess the risk they >>> take by using a feature, without always requiring feature flags. >>> Especially for relatively small features like atan2. There is no >>> intention to drop atan2 in Prometheus 2.x anyway, just we might find a >>> better way to call it. >>> >>> I try to draw a line between what's a useful feature flag, and where >>> just marking experimental in documentation is fine. Prometheus is very >>> conservative anyway, and I value the continuity of our features, >>> including the "experimental" ones. >>> >>> Just to give you an idea, if we had a very strong feature flags policy >>> in Prometheus, here is what it could have looked like, based on >>> >>> https://prometheus.io/docs/prometheus/latest/stability/#api-stability-guarantees >>> >>> --enable-feature=promql-at-modifier >>> --enable-feature=expand-external-labels >>> --enable-feature=promql-negative-offset >>> --enable-feature=remote-write-receiver >>> --enable-feature=exemplar-storage >>> --enable-feature=body-size-limit >>> --enable-feature=relabel-intervals >>> --enable-feature=remote-read >>> --enable-feature=https-basic-auth >>> --enable-feature=web-ui >>> --enable-feature=service-discovery-k8s >>> --enable-feature=service-discovery-consul >>> --enable-feature=remote-write-retry-on-429 >>> --enable-feature=target-limit >>> >>> And that's what I want to avoid. >>> >>> >>> Concretely, my proposal is to continue to be able to mark features as >>> experimental in the documentation, without requiring feature flags. >>> Feature flags can be introduced when some conditions are met, to the >>> appreciation of the maintainers. In some cases (breaking changes or >>> extra unexpected resource consumption), they are mandatory. >>> >>> -- >>> Julien Pivotto >>> @roidelapluie >>> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/564a3e70-0f3a-44b4-88c4-1a1bc70d49f9n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-developers/564a3e70-0f3a-44b4-88c4-1a1bc70d49f9n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0rqLxzJDBsVy2amCStjTBhU7J%3DES9_nP70ED4E-5sAAug%40mail.gmail.com.
