Hi Zhanghaobo, Thanks for the message.
No I don't use as service, as I said the command line is the following: hdfs balancer -Ddfs.balancer.movedWinWidth=5400000 -Ddfs.balancer.moverThreads=1000 -Ddfs.balancer.dispatcherThreads=200 -Ddfs.datanode.balance.max.concurrent.moves=50 -Ddfs.datanode.balance.bandwidthPerSec=100m -Ddfs.balancer.max-size-to-move=10737418240 -threshold 1 Also no other balancer is running concurrently on any other node. Sébastien Le dim. 9 mars 2025 à 13:57, Zhanghaobo <[email protected]> a écrit : > > Hi, @Sébastien Rebecchi > Don't know more details about how you start balancer, did you use > -asService? > > > ---- Replied Message ---- > From Sébastien Rebecchi<[email protected]> > <[email protected]> > Date 3/9/2025 18:03 > To <[email protected]>, > <[email protected]><[email protected]> > <[email protected]> > Subject Re: Can not run HDFS balancer cause metrics already exists > Hello > > Could anyone help on this please? > Situation is still the same after several days. > I add some precisions > - hadoop version 3.4.1 > - balancer command line run: hdfs balancer > -Ddfs.balancer.movedWinWidth=5400000 -Ddfs.balancer.moverThreads=1000 > -Ddfs.balancer.dispatcherThreads=200 > -Ddfs.datanode.balance.max.concurrent.moves=50 > -Ddfs.datanode.balance.bandwidthPerSec=100m > -Ddfs.balancer.max-size-to-move=10737418240 -threshold 1 > > Thank you > > > Le mar. 4 mars 2025, 16:59, Sébastien Rebecchi <[email protected]> > a écrit : > >> Hello >> >> After having added a new node on my HDFS cluster, I try running balancer, >> but it always fails with the following error, even after retrying >> multiple times during the day, and even after having restarted name node >> What should I do to unlock? >> >> Thanks, >> >> Sébastien >> >> >> ERROR balancer.Balancer: Exiting balancer due an exception >> org.apache.hadoop.metrics2.MetricsException: Metrics source >> Balancer-{HERE REPLACE BY CLUSTER'S BLOCK POOL ID} already exists! >> at >> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152) >> at >> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125) >> at >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229) >> at >> org.apache.hadoop.hdfs.server.balancer.BalancerMetrics.create(BalancerMetrics.java:52) >> at >> org.apache.hadoop.hdfs.server.balancer.Balancer.<init>(Balancer.java:362) >> at >> org.apache.hadoop.hdfs.server.balancer.Balancer.doBalance(Balancer.java:824) >> at >> org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:868) >> at >> org.apache.hadoop.hdfs.server.balancer.Balancer$Cli.run(Balancer.java:975) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82) >> at >> org.apache.hadoop.hdfs.server.balancer.Balancer.main(Balancer.java:1133) >> >
