[
https://issues.apache.org/jira/browse/HADOOP-17893?focusedWorklogId=652326&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-652326
]
ASF GitHub Bot logged work on HADOOP-17893:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 17/Sep/21 14:07
Start Date: 17/Sep/21 14:07
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3426:
URL: https://github.com/apache/hadoop/pull/3426#issuecomment-921825663
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|:----:|----------:|--------:|:--------:|:-------:|
| +0 :ok: | reexec | 26m 15s | | Docker mode activated. |
|||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
|||| _ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 7s | | trunk passed |
| +1 :green_heart: | compile | 25m 27s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 19m 57s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 31s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 2m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 8s | | branch has no errors
when building and testing our client artifacts. |
|||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 1s | | the patch passed |
| +1 :green_heart: | compile | 26m 27s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 26m 27s | | the patch passed |
| +1 :green_heart: | compile | 32m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 32m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 48s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 43s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 2m 42s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 16s | | the patch passed |
| -1 :x: | shadedclient | 30m 38s | | patch has errors when building
and testing our client artifacts. |
|||| _ Other Tests _ |
| -1 :x: | unit | 1m 43s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3426/2/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch failed. |
| +0 :ok: | asflicense | 0m 42s | | ASF License check generated no
output? |
| | | 245m 7s | | |
| Reason | Tests |
|-------:|:------|
| Failed junit tests | hadoop.io.TestBloomMapFile |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3426/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3426 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 91a9b65ce965 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29
06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dfc535e426cf05342d371905709205ab08ea9ade |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3426/2/testReport/ |
| Max. process+thread count | 520 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3426/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 652326)
Time Spent: 1h 10m (was: 1h)
> Improve PrometheusSink for Namenode TopMetrics
> ----------------------------------------------
>
> Key: HADOOP-17893
> URL: https://issues.apache.org/jira/browse/HADOOP-17893
> Project: Hadoop Common
> Issue Type: Improvement
> Components: metrics
> Affects Versions: 3.4.0
> Reporter: Max Xie
> Assignee: Max Xie
> Priority: Minor
> Labels: pull-request-available
> Attachments: HADOOP-17893.01.patch
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> HADOOP-16398 added exporter for hadoop metrics to prometheus. But some of
> metrics can't be exported validly. For example like these metrics,
> 1. -queue metrics for ResourceManager-
> {code:java}
> queue_metrics_max_capacity{queue="root.queue1",context="yarn",hostname="rm_host1"}
> 1
> // queue2's metric can't be exported
> queue_metrics_max_capacity{queue="root.queue2",context="yarn",hostname="rm_host1"}
> 2
> {code}
> -It always exported only one queue's metric because
> PrometheusMetricsSink$metricLines only cache one metric if theses metrics
> have the same name no matter these metrics has different metric tags.-
>
> 2. -rpc metrics for Namenode-
> -Namenode may have rpc metrics with multi port like service-rpc. But because
> the same reason as Issue 1, it wiil lost some rpc metrics if we use
> PrometheusSink.-
> {code:java}
> rpc_rpc_queue_time300s90th_percentile_latency{port="9000",servername="ClientNamenodeProtocol",context="rpc",hostname="nnhost"}
> 0
> // rpc port=9005 metric can't be exported
> rpc_rpc_queue_time300s90th_percentile_latency{port="9005",servername="ClientNamenodeProtocol",context="rpc",hostname="nnhost"}
> 0
> {code}
> 3. TopMetrics for Namenode
> org.apache.hadoop.hdfs.server.namenode.top.metrics.TopMetrics is a special
> metric. And I think It is essentially a Summary metric type. TopMetrics
> record name will according to different user and op , which means that these
> metric will always exist in PrometheusMetricsSink$metricLines and it may
> cause the risk of its memory leak. We e need to treat it special.
> {code:java}
> // invaild topmetric export
> # TYPE
> nn_top_user_op_counts_window_ms_1500000_op_safemode_get_user_hadoop_client_ip_test_com_count
> counter
> nn_top_user_op_counts_window_ms_1500000_op_safemode_get_user_hadoop_client_ip_test_com_count{context="dfs",hostname="nn_host",op="safemode_get",user="hadoop/[email protected]"}
> 10
> // it should be
> # TYPE nn_top_user_op_counts_window_ms_1500000_count counter
> nn_top_user_op_counts_window_ms_1500000_count{context="dfs",hostname="nn_host",op="safemode_get",user="hadoop/[email protected]"}
> 10{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
