Hi Chris, thanks a lot for your response. I definitively didn't understand that line of the documentation.
I just tested the solution on the simple test and it works as expected. Thanks a lot, Esau. On Wednesday, April 7, 2021 at 5:00:11 PM UTC+1 [email protected] wrote: > Hello, > > It appears that there is a subtle bug/misunderstanding in the code that is > linked, though that is possibly due to the multiprocess documentation not > being clear enough. When the code specifies a registry for each metric ( > example > <https://github.com/amitsaha/python-prometheus-demo/blob/master/flask_app_prometheus_multiprocessing/src/helpers/middleware.py#L17>), > > it is causing both a process-local metric, and the multi process metrics to > be registered, so depending on which process handles the request you will > get different responses for the metric with "Request latency" in the HELP > text. In the multiprocess documentation > <https://github.com/prometheus/client_python/#multiprocess-mode-gunicorn> > this is what is meant by "Registries can not be used as normal, all > instantiated metrics are exported". If you remove the registry=registry > lines in the example you will see just the multiprocess output as expected. > > You could also move the registry and MultiProcessCollector code into the > request handler to make it clear that the registry used by the > MultiProcessCollector should not have anything registered to it, as seen in > the example in the multiprocess documentation I linked above. > > Let me know if that was unclear or you have more questions, > Chris > > On Wed, Apr 7, 2021 at 8:12 AM Esau Rodriguez <[email protected]> wrote: > >> Hi all, >> I'm not sure if I'm missing something but I'm seeing a behaviour with the >> python client and multiprocess using gunicorn and flask I'm not sure if I'm >> missing something or there's a bug there. >> >> When I hit the endpoint producing the prometheus text to be scrapped I'm >> seeing 2 versions for the same metrics with different help texts. I would >> expect to see only one metric (the multiprocess one). >> >> I thought I had something wrong in my setup so I tried it with e pretty >> simple project that I found here >> https://github.com/amitsaha/python-prometheus-demo/tree/master/flask_app_prometheus_multiprocessing >> >> (not my code). >> >> I hit a random url and then the `/metrics` endpoint >> >> You can see in the raw response down here we have 2 entries for each >> metric, with different `types` and `help` texts. In this example there >> really wasn't any processes but in the real example in prod we have several >> processes and we see the prometheus scraper `picks` a different value >> depending on the order of the response. >> >> Am I missing something or is there a bug there? >> >> The raw response was: >> >> <pre> >> % curl --location --request GET 'http://localhost:5000/metrics' >> # HELP request_latency_seconds Multiprocess metric >> # TYPE request_latency_seconds histogram >> request_latency_seconds_sum{app_name="webapp",endpoint="/metrics"} >> 0.00040912628173828125 >> request_latency_seconds_sum{app_name="webapp",endpoint="/"} >> 0.0001652240753173828 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.005"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.01"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.025"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.05"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.075"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.1"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.25"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.5"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.75"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="1.0"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="2.5"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="5.0"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="7.5"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="10.0"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="+Inf"} >> >> 1.0 >> request_latency_seconds_count{app_name="webapp",endpoint="/metrics"} 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.005"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.01"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.025"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.05"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.075"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.1"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.25"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.5"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.75"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="1.0"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="2.5"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="5.0"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="7.5"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="10.0"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="+Inf"} >> 1.0 >> request_latency_seconds_count{app_name="webapp",endpoint="/"} 1.0 >> # HELP request_count_total Multiprocess metric >> # TYPE request_count_total counter >> request_count_total{app_name="webapp",endpoint="/metrics",http_status="200",method="GET"} >> >> 1.0 >> request_count_total{app_name="webapp",endpoint="/",http_status="404",method="GET"} >> >> 1.0 >> # HELP request_count_total App Request Count >> # TYPE request_count_total counter >> request_count_total{app_name="webapp",endpoint="/metrics",http_status="200",method="GET"} >> >> 1.0 >> request_count_total{app_name="webapp",endpoint="/",http_status="404",method="GET"} >> >> 1.0 >> # HELP request_count_created App Request Count >> # TYPE request_count_created gauge >> request_count_created{app_name="webapp",endpoint="/metrics",http_status="200",method="GET"} >> >> 1.617798968564061e+09 >> request_count_created{app_name="webapp",endpoint="/",http_status="404",method="GET"} >> >> 1.61779898142748e+09 >> # HELP request_latency_seconds Request latency >> # TYPE request_latency_seconds histogram >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.005"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.01"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.025"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.05"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.075"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.1"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.25"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.5"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="0.75"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="1.0"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="2.5"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="5.0"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="7.5"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="10.0"} >> >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/metrics",le="+Inf"} >> >> 1.0 >> request_latency_seconds_count{app_name="webapp",endpoint="/metrics"} 1.0 >> request_latency_seconds_sum{app_name="webapp",endpoint="/metrics"} >> 0.00040912628173828125 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.005"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.01"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.025"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.05"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.075"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.1"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.25"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.5"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="0.75"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="1.0"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="2.5"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="5.0"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="7.5"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="10.0"} >> 1.0 >> request_latency_seconds_bucket{app_name="webapp",endpoint="/",le="+Inf"} >> 1.0 >> request_latency_seconds_count{app_name="webapp",endpoint="/"} 1.0 >> request_latency_seconds_sum{app_name="webapp",endpoint="/"} >> 0.0001652240753173828 >> # HELP request_latency_seconds_created Request latency >> # TYPE request_latency_seconds_created gauge >> request_latency_seconds_created{app_name="webapp",endpoint="/metrics"} >> 1.617798968520208e+09 >> request_latency_seconds_created{app_name="webapp",endpoint="/"} >> 1.617798981426993e+09 >> </pre> >> >> Kind regards, >> Esau. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Prometheus Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/dd6d0ea6-ef26-4aaf-b926-9490a109c15cn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/prometheus-developers/dd6d0ea6-ef26-4aaf-b926-9490a109c15cn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/d33a0134-0033-434e-a268-8ccaf9984b3en%40googlegroups.com.
