Thank you for the response Brian, I removed the $__ variables and tried viewing disk usage metrics from past 1 hour in PromUI -
I tried the query in the Prometheus UI , and i was expecting value ~97% for past 1 hour metrics but the table view reports 48%. [image: max_over_time.png] I am not sure if i missed out on something within the query. i am under impression that max function works with multiple series, and over time will generate stats from the values within the series. Please advice. On Tuesday, September 24, 2024 at 8:12:29 PM UTC+5:30 Brian Candler wrote: > $__rate_interval is (roughly speaking) the interval between 2 adjacent > points in the graph, with a minimum of 4 times the configured scrape > interval. It's not the entire period over which Grafana is drawing the > graph. You probably want $__range or $__range_s. See: > > https://grafana.com/docs/grafana/latest/datasources/prometheus/template-variables/#use-__rate_interval > > https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#global-variables > > However, questions about Grafana would be better off asked in the Grafana > community. Prometheus is not Grafana, and those variables are > Grafana-specific. > > > so you can see that avg|min|max_over_time functions return identical > values which dont make much sense > > It makes sense when you realise that the time period you're querying over > is very small; hence for a value that doesn't change rapidly, the > min/max/average over such a short time range will all be roughly the same. > > On Tuesday 24 September 2024 at 15:10:33 UTC+1 mohan garden wrote: > >> Hi , >> seems images in my previous post did not show up as expected. >> Sorry for the spam , reposting again - >> >> >> Hi , >> >> I am trying to analyse memory usage of a server for 2 specific months >> using Grafana and prometheus. but seems _over_time functions are returning >> unexpected results. >> >> Here is the data for the duration >> >> [image: one.png] >> >> Now, the summary table shows expected values >> [image: one.png] >> >> >> query - >> (( >> node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> * 100 ) / >> node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval] >> >> >> Issue - when i am trying to create similar stats using PromQL at my end , >> i am facing issues . i fail to get the same stats when i use the following >> promql , example - >> >> [image: two.png] >> >> ( >> avg_over_time(node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> / >> avg_over_time(node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> ) * 100 >> >> ( >> min_over_time(node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> / >> min_over_time(node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> ) * 100 >> >> ( >> max_over_time(node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> / >> max_over_time(node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval]) >> >> ) * 100 >> >> so you can see that avg|min|max_over_time functions return identical >> values which dont make much sense. I was using following setting >> >> [image: one.png] >> >> I tried changing from range -> instant, i see similar values >> [image: two.png] >> >> Where do i need to make modifications in PromQL so i can get the correct >> min/max/avg values in the gauges as correctly reported by the >> [image: one.png] >> >> >> for a specific duration , say - >> >> [image: one.png] >> >> please advice >> >> >> >> >> >> >> >> >> >> On Tuesday, September 24, 2024 at 7:25:00 PM UTC+5:30 mohan garden wrote: >> >>> I am trying to analyse memory usage of a server for 2 specific months >>> using Grafana and prometheus. but seems _over_time functions are returning >>> unexpected results. >>> >>> Here is the data for the duration >>> [image: image] >>> <https://private-user-images.githubusercontent.com/154580421/370293312-9755f80b-01f4-4785-8d94-5d4e55b3cb0b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii8xNTQ1ODA0MjEvMzcwMjkzMzEyLTk3NTVmODBiLTAxZjQtNDc4NS04ZDk0LTVkNGU1NWIzY2IwYi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTI0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkyNFQxMzQ4NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1iNmUzNjQ2Yzk5MjllZWQwYjhjOWZjZjk4MTNmYTFjN2ZkNDE3YzA2NDRiOWZiMzdkY2FkYWYyZGJkNDU4M2ZiJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.bVjNK3XE9QLGybgL6GUbJ-VHvUMk-0m4yJIsHEAeGo4> >>> the summary table shows expected values >>> [image: image] >>> <https://private-user-images.githubusercontent.com/154580421/370293517-6a3d94db-f0eb-4915-a0d5-b9d235e5a944.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii8xNTQ1ODA0MjEvMzcwMjkzNTE3LTZhM2Q5NGRiLWYwZWItNDkxNS1hMGQ1LWI5ZDIzNWU1YTk0NC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTI0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkyNFQxMzQ4NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1kMzVhNTE5ZTQ1Nzg2MDhmMDcyYWI1YjI1MGQ0YTkwMDZhNDJlYjYyODQ2Mjg1MWRiNWJkNGVmOTMyNjE0MjA1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.POaO2I4CLHlxSH9gzIbF8W2Hl3mAcorBj_FjQlR4SOQ> >>> query - >>> (( >>> node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> * 100 ) / >>> node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval] >>> >>> >>> Issue - when i am trying to create similar stats using PromQL at my end >>> , i am facing issues . i fail to get the same values when i use promql , >>> example - >>> >>> [image: image] >>> <https://private-user-images.githubusercontent.com/154580421/370295234-12eeea9c-a974-4a08-b21a-bb9b6bfbfc87.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii8xNTQ1ODA0MjEvMzcwMjk1MjM0LTEyZWVlYTljLWE5NzQtNGEwOC1iMjFhLWJiOWI2YmZiZmM4Ny5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTI0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkyNFQxMzQ4NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT01NzY2MjE1MzNmMmM0ZjRkZWYyYWU3Y2ViOTFjNTk3NTJlYjNlN2Q3OTMzMTY0MzhiOTllODY2ZDIxNzExYjVkJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.WPjDSluZWsNcTbCVDVnoTB3wa3CSdFgBXRq7bxu5HS0> >>> >>> ( >>> avg_over_time(node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> / >>> avg_over_time(node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> ) * 100 >>> >>> ( >>> min_over_time(node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> / >>> min_over_time(node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> ) * 100 >>> >>> ( >>> max_over_time(node_memory_MemAvailable_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> / >>> max_over_time(node_memory_MemTotal_bytes{instance="$node",job="$job"}[$__rate_interval]) >>> >>> ) * 100 >>> >>> so you can see that avg|min|max_over_time functions return identical >>> values with following setting >>> [image: image] >>> <https://private-user-images.githubusercontent.com/154580421/370295926-18b0a6a3-fea5-48b9-8f09-715df5725f34.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii8xNTQ1ODA0MjEvMzcwMjk1OTI2LTE4YjBhNmEzLWZlYTUtNDhiOS04ZjA5LTcxNWRmNTcyNWYzNC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTI0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkyNFQxMzQ4NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT03MjI5MjI5NDNhODA2ZjhkNjIwNTA3ZjAwM2VlNTlmM2I4NWM5ZjdiNGM5NjExYmE1ZTg2Njg2NzllOTM5YWFjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.Nbpvcc5P80H3yC3uOHnYHGb7FwpbmMLV4xujhJ6YqR0> >>> >>> when i change from range -> instant, i see similar values >>> [image: image] >>> <https://private-user-images.githubusercontent.com/154580421/370296399-fbba4ab2-7efb-40d0-a067-3705ddbc111e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii8xNTQ1ODA0MjEvMzcwMjk2Mzk5LWZiYmE0YWIyLTdlZmItNDBkMC1hMDY3LTM3MDVkZGJjMTExZS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTI0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkyNFQxMzQ4NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wYzk1OTczOGVmMGVlNWFjYjY2NDkwMzQ0MzVjNzc5NzI3YzcwNzk3MmRkYTJlMzhiMzJhNDNmNzI0NjlkYTI1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.-q5VtuzgNYlX5_r9nui9UJWLyFUTMU3H7Q8I7l0giFk> >>> >>> Where do i need to make modifications in PromQL so i can get the correct >>> min/max/avg values in the gauges as reported by the >>> [image: image] >>> <https://private-user-images.githubusercontent.com/154580421/370293517-6a3d94db-f0eb-4915-a0d5-b9d235e5a944.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii8xNTQ1ODA0MjEvMzcwMjkzNTE3LTZhM2Q5NGRiLWYwZWItNDkxNS1hMGQ1LWI5ZDIzNWU1YTk0NC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwOTI0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDkyNFQxMzQ4NDNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1kMzVhNTE5ZTQ1Nzg2MDhmMDcyYWI1YjI1MGQ0YTkwMDZhNDJlYjYyODQ2Mjg1MWRiNWJkNGVmOTMyNjE0MjA1JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCJ9.POaO2I4CLHlxSH9gzIbF8W2Hl3mAcorBj_FjQlR4SOQ> >>> for a specific duration , say - >>> [image: image] >>> <https://private-user-images.githubusercontent.com/5935825/370298748-b4ecd557-f74d-44da-a33c-3df91a8c199d.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjcxODYwMjMsIm5iZiI6MTcyNzE4NTcyMywicGF0aCI6Ii81OTM1ODI1LzM3MDI5ODc0OC1iNGVjZDU1Ny1mNzRkLTQ0ZGEtYTMzYy0zZGY5MWE4YzE5OWQucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkyNCUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MjRUMTM0ODQzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjVhZWJhNGM1MjE1MTUwODc5OTgxMmVhOGU2MDJhZTI5Y2EwNWQ3YmRhODRmN2I3MzRjN2E4ZTQzZTQ1MTgyOSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.Qw9GrH1ef_Gj1jAUNtYT2Bz05xZ1YUtVELfYJJP2Cds> >>> >>> please advice >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/8419c0b2-0826-47d9-ac1b-c9c9f0bc1f6bn%40googlegroups.com.
