Sorry, unclear phrasing – I mean as the one scraping (whether by hand or as a Prometheus server), how would I know what the semantics of the value in the provided exemplar are, whether they are a delta or an absolute value? Again, I think "it's always an absolute value" makes this least surprising.
This overlaps with the discussion we had about exemplars from recording rules – if there was the option to have multiple exemplars for one gauge, it could separately present exemplars for increment, decrement, and set, whichever ones have been used … /MR On Wed, Dec 7, 2022 at 9:50 AM 'Fabian Stäber' via Prometheus Developers < [email protected]> wrote: > > At scrape time, how would I know which method was used? > > The library keeps a list of Exemplars that are provided at scrape time (in > case of a Gauge the list only has 1 element, because Gauges only have 1 > Exemplar). > > The question is what to put in the list when inc(), dec(), or set() are > called. > > My feeling is also that the total value of the gauge is least surprising. > But that would also mean that for cases like "number of Bytes in a queue" > you'd not see the number of Bytes added as an Exemplar value, just the > total queue size. > > Fabian > > On Wed, Dec 7, 2022 at 10:44 AM Matthias Rampke <[email protected]> > wrote: > >> At scrape time, how would I know which method was used? "always the total >> value of the gauge" seems like the least surprising choice to me. If users >> need to track separate exemplars for increment and decrement, they could >> use two counters (basically, a non-native UpDownCounter), which >> conceptually and physically preserves that the final value is the result of >> repeated additions and removals. >> >> /MR >> >> On Wed, Dec 7, 2022 at 7:58 AM 'Fabian Stäber' via Prometheus Developers < >> [email protected]> wrote: >> >>> Thanks Julien. I guess there are examples for both, having Exemplars >>> with the delta as well as the absolute value. >>> >>> Example 1) Number of Bytes in a queue. You would call >>> myGauge.inc(nBytes) if a message is added to the queue, and >>> myGauge.dec(nBytes) if a message is fetched from the queue. If adding / >>> fetching is done via REST API, it would make sense to have Exemplars for >>> the add/fetch operations. >>> >>> Example 2) An IoT device reaching out to a Web hook for reporting >>> temperature. To track the current temperature, you'd call >>> myGauge.set(temperature). If the IoT device is capable of adding trace >>> headers when calling the Web hook it would be good to have Exemplars of >>> these calls. >>> >>> The Java library supports automatic Exemplars, i.e. users don't call >>> explicit API to add Exemplars, but the library figures out if a current >>> trace context exists and samples Exemplars automatically under the hood. >>> For that, we need a default behavior. So what would a good default be? >>> Always the total value of the Gauge, or always the delta, or make it depend >>> on whether inc(), dec(), or set() was called? >>> >>> Fabian >>> >>> >>> On Tue, Dec 6, 2022 at 11:18 PM Julien Pivotto < >>> [email protected]> wrote: >>> >>>> On 06 Dec 23:06, 'Fabian Stäber' via Prometheus Developers wrote: >>>> > Hi, >>>> > >>>> > I'm experimenting with Exemplars for Gauge metrics in client_java >>>> > (background: at the dev summit on 10 November 2022 >>>> > < >>>> https://docs.google.com/document/d/11LC3wJcVk00l8w5P3oLQ-m3Y37iom6INAMEu2ZAGIIE/edit >>>> > >>>> > we decided that "Prometheus will ingest Exemplars on all time >>>> series"). >>>> > >>>> > For comparison: A counter MUST have the following methods >>>> > <https://prometheus.io/docs/instrumenting/writing_clientlibs/#counter >>>> >: >>>> > >>>> > - inc(): Increment the counter by 1 >>>> > - inc(double v): Increment the counter by the given amount. MUST >>>> check >>>> > that v >= 0. >>>> > >>>> > Exemplars will contain the increment, i.e. if inc() is called the >>>> Exemplar >>>> > will have the value 1.0, if inc(v) is called the Exemplar will have >>>> the >>>> > value v. >>>> > >>>> > Now, A gauge MUST have the following methods >>>> > <https://prometheus.io/docs/instrumenting/writing_clientlibs/#gauge>: >>>> > >>>> > - inc(): Increment the gauge by 1 >>>> > - inc(double v): Increment the gauge by the given amount >>>> > - dec(): Decrement the gauge by 1 >>>> > - dec(double v): Decrement the gauge by the given amount >>>> > - set(double v): Set the gauge to the given value >>>> > >>>> > Which value should we choose for Gauge Exemplars? >>>> >>>> That is a good question and my guess would be that it will depend on the >>>> use case. Do you have examples of gauges would would attach exemplars >>>> to? >>>> >>>> -- >>>> Julien Pivotto >>>> @roidelapluie >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-developers/CAPX310ipsufix2s4_P4PXOvzGr4MFDbHaaBmmJZXpnW-k%2Bas6g%40mail.gmail.com >>> <https://groups.google.com/d/msgid/prometheus-developers/CAPX310ipsufix2s4_P4PXOvzGr4MFDbHaaBmmJZXpnW-k%2Bas6g%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CAPX310ggv%2BthbzFYx5j71B%2ByyOOy9d-64nPAa00-yNd6jcZbkQ%40mail.gmail.com > <https://groups.google.com/d/msgid/prometheus-developers/CAPX310ggv%2BthbzFYx5j71B%2ByyOOy9d-64nPAa00-yNd6jcZbkQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAMV%3D_gaT6_kDoK0TY6QvnMYZ5hrGiSAF_ee2eBgOBA5UuOQ1Ow%40mail.gmail.com.
