Oh I see. I agree with that idea, I'm gonna move this discussion to their github issues.
Thank you! Em quarta-feira, 31 de maio de 2023 às 11:43:41 UTC-3, Julius Volz escreveu: > That's the TSDB complaining about a related problem where multiple series > with the same label identity are likely coming in. But that shouldn't lead > to PromQL queries receiving the same series multiple times in a vector > selector unless something is broken in the TSDB / Thanos data merging layer > underneath PromQL. Could you share all the details of your Thanos setup > (how many Prometheus servers with which external labels do you have, how > are you merging them together with Thanos, which Thanos and Prometheus > versions are you using, are you using object storage for long-term data, > etc.)? Might at some point be a better question / bug report for the Thanos > folks though. > > On Wed, May 31, 2023 at 3:07 PM Adso “Cho” Castro <[email protected]> > wrote: > >> Hey Julius, morning >> >> Found a few warn logs from thanos-receive, maybe a possible root cause? >> >> - "Error on ingesting samples with different values but same timestamp" >> - "Error on ingesting out-of-order samples" >> >> >> >> Em terça-feira, 30 de maio de 2023 às 18:02:26 UTC-3, Julius Volz >> escreveu: >> >>> Nothing in the query itself should be able to cause a duplicate >>> labelset in a normal PromQL execution, so I assume that something is wrong >>> with the underlying Thanos setup or code. Maybe it somehow manages to >>> select the same duplicate time series from multiple Prometheus servers (or >>> the long-term storage), because something around the external labels >>> normally used for differentiating series isn't configured or working >>> properly? >>> >>> On Tue, May 30, 2023 at 10:28 PM Adso “Cho” Castro <[email protected]> >>> wrote: >>> >>>> Hey, been getting the error below when querying: >>>> >>>> *execution: vector cannot contain metrics with the same labelset* >>>> >>>> The query: >>>> >>>> sum(rate(istio_requests_total{reporter="source", >>>> destination_workload="dummy-data", response_code=~"2.."}[$_range])) by >>>> (destination_workload) / sum(rate(istio_requests_total{reporter="source", >>>> destination_workload="dummy-data"}[$_range])) by (destination_workload) >>>> >>>> Funny thing is, this is a Thanos installation. I have a Cortex >>>> installation with the same data and this query works fine. >>>> >>>> Thanks >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Prometheus Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/prometheus-users/b08f43a8-fd11-4775-838e-350f144df8e8n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/prometheus-users/b08f43a8-fd11-4775-838e-350f144df8e8n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> >>> >>> -- >>> Julius Volz >>> PromLabs - promlabs.com >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/36531430-934f-4334-a78c-66abb1b16aa7n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-users/36531430-934f-4334-a78c-66abb1b16aa7n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > > > -- > Julius Volz > PromLabs - promlabs.com > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/00f119bd-651f-4d5c-843a-5c16c0102c16n%40googlegroups.com.
