> just I am a bit concerned about one binary that enables everything. It might be quite hard work to maintain it.
vmagent is just easy to use metrics proxy, which performs the following tasks: - Accepts data via various popular ingestion protocols (Prometheus remote_write, Influx line protocol, Graphite, OpenTSDB, CSV). Additionally, it can scrape Prometheus targets. - Augments and filters the accepted data with Prometheus-compatible relabeling. - Pushes the filtered data to the configured remote storage targets via Prometheus remote_write protocol. Additional per-target relabeling can be applied to data before sending it to each remote storage target. vmagent uses independent file-base buffers for each configured remote storage target, so it may buffer the data locally until temporarily unavailable remote storage target becomes available again. On Mon, Apr 20, 2020 at 9:44 PM Bartłomiej Płotka <[email protected]> wrote: > Thanks, Aliaksandr! > > So vmagent on top of scraping ALSO receive remote write API? 😱 What it > CANNOT do? =D > > It looks indeed that feature-wise it is what we meant, just I am a bit > concerned about one binary that enables everything. It might be quite hard > work to maintain it... You must be some kind of superhuman Aliaksandr! (: > Definitely will take a look, thanks. (: > > Kind Regards, > Bartek > > On Mon, 20 Apr 2020 at 19:38, Aliaksandr Valialkin <[email protected]> > wrote: > >> Such a mirroring can be done with vmagent >> <https://github.com/VictoriaMetrics/VictoriaMetrics/blob/master/app/vmagent/README.md> >> - just configure multiple `-remoteWrite.url` targets with distinct >> `-remoteWrite.urlRelabelConfig` configs for obfuscation and filtering. The >> final system will look like the following: >> >> -> remote target1 (prod) >> Prometheus -> vmagent -> filtering -> remote target2 (staging) >> -> obfuscation -> remote target3 (dev) >> >> >> On Mon, Apr 20, 2020 at 9:24 PM Bartłomiej Płotka <[email protected]> >> wrote: >> >>> Hi! >>> >>> This question is not strictly related to Prometheus, but rather to >>> server-side Remote Write APIs. >>> We are looking at how to have more realistic staging environments for >>> servers like that. In order to achieve so, we want to "mirror" / "fork" >>> portion of production remote write traffic to other clusters APIs (e.g >>> staging or dev environment). >>> >>> As part of this mirroring, data has to be potentially obfuscated to >>> avoid leaking of sensitive data, but also without totally changing the >>> characteristic of data (e.g same number labels, labels values/names with >>> the same amount of characters, etc). >>> >>> In the future, we could add some more advanced features if needed (e.g >>> load balancing). >>> >>> Wonder if anyone in the community had been working on something like >>> that already and has something to share/is already shared? >>> >>> ProxySQL <https://github.com/sysown/proxysql/wiki/Mirroring> is >>> something like that but in the SQL world. Would be awesome to have the same >>> for remote write (and Query API as well I guess, but let's think about it >>> in a separate thread) (: >>> <https://github.com/thanos-io/thanos/issues/2480> >>> Some discussion on Thanos project: >>> https://github.com/thanos-io/thanos/issues/2480 >>> >>> Please help if you know or have worked on something like this (: Would >>> be a nice community Project if nothing exists! >>> >>> Kind Regards, >>> Bartek >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/CAMssQwYa3kW8UMPtJ2PuW8%3Dd8kWB-sz1E99D20ODn28KZTb%2BNQ%40mail.gmail.com >>> <https://groups.google.com/d/msgid/prometheus-users/CAMssQwYa3kW8UMPtJ2PuW8%3Dd8kWB-sz1E99D20ODn28KZTb%2BNQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Best Regards, >> >> Aliaksandr >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-users/CAPbKnmBEvDbThrr1cx9FgPhZRVrJqeS2%3DsgEOZguoMWH6FS7BA%40mail.gmail.com >> <https://groups.google.com/d/msgid/prometheus-users/CAPbKnmBEvDbThrr1cx9FgPhZRVrJqeS2%3DsgEOZguoMWH6FS7BA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- Best Regards, Aliaksandr -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAPbKnmB21iwN2qTWFyZDOA12Ot5a5_02Cn0TOnxtANBxdHdf1Q%40mail.gmail.com.
