Hi, You can use JMX exporter that gives you a protheus target, this https://github.com/prometheus/jmx_exporter. will help in your case.
On Wed, Jun 3, 2020 at 3:33 PM Sanel Zukan <[email protected]> wrote: > ravi kanth <[email protected]> writes: > > Wei-Chiu, > > > > Thanks for the response. Are there any workarounds for Hadoop 3.1.0? As > > this is a critical production cluster it needs immediate attention. > > My approach was a bit different. > > Instead of using Prometheus to pull the metrics, I used collectd [1] to > collect system and JMX metrics, send that to Riemann [2] and store it in > database (in my case OpenTSDB, but changing it to Prometheus pushgateway > address would be couple of lines of code in Riemann). > > Inside Riemann, you can tag, shape and calculate metrics the way you > like. > > There is also standalone app [3] you can run to watch for multiple JMX > connections, but I haven't tried that. > > [1] https://collectd.org/ > [2] http://riemann.io/ > [3] https://github.com/twosigma/riemann-jmx > > > Thanks, > > Ravi > > Best, > Sanel > > > On Tue, Jun 2, 2020 at 5:59 PM Wei-Chiu Chuang <[email protected]> > wrote: > > > >> Check out HADOOP-16398 > >> <https://issues.apache.org/jira/browse/HADOOP-16398> > >> It's a new feature in Hadoop 3.3.0 > >> > >> Akira might be able to help. > >> > >> On Tue, Jun 2, 2020 at 5:56 PM ravi kanth <[email protected]> wrote: > >> > >>> Hi Everyone, > >>> > >>> We have a production-ready cluster with 35 nodes that we are currently > >>> using. We are currently using System metrics using Prometheus + > Grafana to > >>> visualize server. However, we are more interested in visualizing the > Hadoop > >>> & Yarn service level metrics. > >>> > >>> I came across hadoop JMX port which exposes all the needed metrics from > >>> the service. However, I remained unsuccessful in tagging these metrics > to > >>> prometheus jmx agent. > >>> > >>> Is there anyone who successfully got the JMX monitoring of Hadoop > >>> components work with Prometheus? Any help is greatly appreciated. > >>> > >>> Currently, we have scripts to parse the meaningful values out of JMX > end > >>> point of the namenode & datanodes. > >>> > >>> Thanks In advance, > >>> > >>> Ravi > >>> > >>> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Thanks & Regards, Venkatesh SRE, Media.net (Data Platform) Flock Id- [email protected] Contact - 9949522101
