The @ modifier is no longer experimental. It was made a permanent part of Prometheus in Jan 2022, when the promql-at-modifier was made a no-op. It is no longer listed under feature flags <https://prometheus.io/docs/prometheus/latest/feature_flags/>.
See commit b39f2739e5b01560ad8299d2579f1041a0e9ae5f <https://github.com/prometheus/prometheus/commit/b39f2739e5b01560ad8299d2579f1041a0e9ae5f> which was included in v2.33.0 On Monday 19 August 2024 at 18:34:48 UTC+1 Simon Hardy-Francis wrote: > I think I found one answer to question 2, which is to turn on this via " > --enable-feature=promql-at-modifier" feature [1]. > However, [2] says such " features .. are disabled by default since they > are breaking changes or are considered experimental". So not sure if I want > to use it. > > [1] https://prometheus.io/blog/2021/02/18/introducing-the-@-modifier/ > [2] https://prometheus.io/docs/prometheus/2.45/feature_flags/ > > On Friday, August 16, 2024 at 9:30:27 PM UTC-7 Simon Hardy-Francis wrote: > >> Hello! >> >> I am a relative newbie to Prometheus and promql and have the following >> questions: >> >> Question 1: I tried to find the CPU used by containers in the last 1 >> minute, using 3 different ways via promql, and the ways do not agree with >> each other. >> >> One way is finding the absolute value for " >> container_cpu_usage_seconds_total" before and after and subtracting the >> values. The other way is using "rate" and the last way is using "increase". >> Why do the "rate" and "increase" ways show so much less CPU being used? >> >> Question 2: Let's say I want to find the CPU used for all containers >> active between 1 and 2 hours ago. And to discover how long each container >> is active for the case that e.g. a container only existed for e.g. 10 >> minutes of that 60 minute window. How to create a promql command to do that? >> >> Thanks, >> Simon >> >> P.S.: Here are my commands: >> >> $ cat promql.try-1m.sh >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' 'sort_desc(sum by (instance, namespace, node, pod, container) >> (container_cpu_usage_seconds_total{container!=""}))' > promql.before.txt) & >> sleep 60 >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' 'sort_desc(sum by (instance, namespace, node, pod, container) >> (container_cpu_usage_seconds_total{container!=""}))' > promql.after.txt) & >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' 'sum(rate(container_cpu_usage_seconds_total{container!=""}[1m])) >> by (node, instance, namespace, pod, container)' > promql.rate-1m.txt) & >> (promql --timeout 300 --no-headers --host 'https://<my prometheus >> server>' >> 'sum(increase(container_cpu_usage_seconds_total{container!=""}[1m])) by >> (node, instance, namespace, pod, container)' > promql.increase-1m.txt) & >> >> $ ./promql.try-1m.sh >> >> $ ls -al promql.*.txt >> -rw-rw-r-- 1 simon simon 3845801 Aug 16 15:36 promql.before.txt >> -rw-rw-r-- 1 simon simon 3844193 Aug 16 15:37 promql.after.txt >> -rw-rw-r-- 1 simon simon 3565377 Aug 16 15:37 promql.increase-1m.txt >> -rw-rw-r-- 1 simon simon 3591045 Aug 16 15:37 promql.rate-1m.txt >> >> $ cat promql.before.txt | egrep kube-router | head -1 >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 826909.35828385 2024-08-16T15:36:16-07:00 >> >> $ cat promql.after.txt | egrep kube-router | egrep >> "kube-router.*kube-router-kjxnp" >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 826929.923827853 2024-08-16T15:37:16-07:00 >> >> $ perl -e 'printf qq[%f\n], 826929.923827853 - 826909.35828385;' >> 20.565544 >> >> $ cat promql.increase-1m.txt | egrep kube-router | egrep >> "kube-router.*kube-router-kjxnp" >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 5.908468843536704 2024-08-16T15:37:16-07:00 >> >> $ cat promql.rate-1m.txt | egrep kube-router | egrep >> "kube-router.*kube-router-kjxnp" >> kube-router 10.34.28.252:10250 kube-system <my k8s node> >> kube-router-kjxnp 0.09846672408107424 2024-08-16T15:37:16-07:00 >> >> $ perl -e 'printf qq[%f\n], 0.09846672408107424 * 60;' >> 5.908003 >> >> $ perl -e 'printf qq[%f%%\n], (20.565544 - 5.908003) / 5.908003 * 100;' >> 248.096370% >> >> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6cd6b961-893c-4394-90ef-14b0ec7c981cn%40googlegroups.com.
