> Thanos automatically deals with downsample lookback with no changes. That's true. We've at M3 have done the same. However, the resulting solutions are not very clean and involve trade-offs (Thanos seem to initialize up to three engine with increasing lookbacks <https://github.com/thanos-io/thanos/blob/243526d6a7108af41ce76586832358c1a467362d/cmd/thanos/query.go#L744-L746>; in M3 we pre-initialize a few engines based on configured resolutions <https://github.com/m3db/m3/blob/4a19fb6da6827d24bd5e7ab7058b5d09b92989f6/src/query/server/query.go#L1399-L1406> and in any other case create them on the fly <https://github.com/m3db/m3/blob/4a19fb6da6827d24bd5e7ab7058b5d09b92989f6/src/query/promqlengine/cache.go#L52-L56>).
>From what we've discussed among M3 maintainers and from TODOs in Thanos codebase <https://github.com/thanos-io/thanos/blob/243526d6a7108af41ce76586832358c1a467362d/cmd/thanos/query.go#L736-L737>, both seem to agree that it would be more convenient if dynamic lookback was supported by Prometheus. > We are willing to have downsampling in Prometheus too in the future. > > I would merge that pull request. I'll look into submitting a draft PR, just so it would be easier to see what changes would be needed. On Monday, November 15, 2021 at 9:12:44 PM UTC+2 [email protected] wrote: > Thanos automatically deals with downsample lookback with no changes. This > was fixed at least 2 years ago. > > On Mon, Nov 15, 2021, 18:54 Julien Pivotto <[email protected]> > wrote: > >> We are willing to have downsampling in Prometheus too in the future. >> >> I would merge that pull request. >> >> Le lun. 15 nov. 2021, 15:19, Vilius Pranckaitis <[email protected]> >> a écrit : >> >>> *TL;DR*: ability to set a custom lookback delta for a query would be >>> useful when time series have drastically different resolutions. >>> >>> --- >>> >>> Currently, Prometheus allows configuring lookback delta through >>> configuration flag. This works well if the resolutions/scrape intervals of >>> time series are similar, but is inconvenient if they differ a lot. E.g. if >>> we had two time series, one with a datapoint each hour and another one with >>> a datapoint each minute, a lookback of 2h would work for the first one, but >>> would be way too big for the second one. >>> >>> This impacts looking at downsampled data greater than 5 minutes (the >>> default lookback period) which both Thanos and M3 frequently keep >>> (depending on user configuration). Both support downsampling data to user >>> configured resolutions, thus lookback delta should be selected according to >>> what resolution is being queried when querying downsampled data. Both >>> Thanos and M3 depend on Prometheus and use `promql.Engine` for executing >>> queries, which doesn't allow setting custom lookback for a particular query. >>> >>> While I understand that this might be more relevant for M3 and Thanos >>> than for Prometheus, just wanted to check what the maintainers of >>> Prometheus think about this. If this seems useful, someone from M3 should >>> be able to submit a PR (we have already looked at the code and it doesn't >>> look like it needs huge changes). >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-developers/84c03b13-25d7-479c-b678-44cd8625f2c0n%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/prometheus-developers/84c03b13-25d7-479c-b678-44cd8625f2c0n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0p-_k_Yjy_gUdvHaogxZz%3D7DeNEMM7zz7uBYD%2BYBA%2B6bw%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0p-_k_Yjy_gUdvHaogxZz%3D7DeNEMM7zz7uBYD%2BYBA%2B6bw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/d1fa16b6-d91c-4fd6-93ef-694af3482e05n%40googlegroups.com.
