Hi there, We have recently deployed the jmx_exporter to monitor our Kafka infrastructure using Prometheus. When initially running the exporter (as a Java agent), we noticed collection time could be as high as 70 seconds on some of our Kafka brokers (partly due to the fact that Kafka exposes a lot of metrics on a per topic-partition basis and we have some large brokers), even after tweaking the configuration to only collect mbeans that were interesting to us.
After digging a bit into the codebase of the jmx_exporter and doing some profiling, we noticed most of the collection time was spent matching mbean names/values against regular expressions <https://github.com/prometheus/jmx_exporter/blob/master/collector/src/main/java/io/prometheus/jmx/JmxCollector.java#L371> in the configuration. We have made changes internally to allow (via a feature-flag) to cache the result of these computations. With caching enabled, collection times went down from ~70 sec to ~3 sec. Note that this prevents having rules that take mbeans values in their pattern from being used (as only mbeans names are cached) I would be happy to open a PR (and discuss potential implementation details) if you think this is a feature worth having in the upstream version of the jmx_exporter? Cheers, Flavien -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/de757ece-f06e-4f4a-beb2-714907c4a3d5n%40googlegroups.com.
