[
https://issues.apache.org/jira/browse/HDFS-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18066057#comment-18066057
]
ASF GitHub Bot commented on HDFS-17891:
---------------------------------------
Jimvin commented on PR #8341:
URL: https://github.com/apache/hadoop/pull/8341#issuecomment-4066772060
This patch was manually tested since there are quite a few moving parts. The
test cases that fail before the patch are as follow:
- `hdfs dfsadmin -report` displays an error
(java.lang.IllegalArgumentException) when showing the datanode report.
- The NameNode web UI is missing status content on the Datanodes and
Datanode Volume Failure tabs.
The test procedure used is as follows:
- Deploy a HDFS cluster on Kubernetes with 3 datanodes deployed to three
separate Kubernetes nodes.
- Cordon one of the Kubernetes nodes and delete the datanode pod run on this
Kubernetes node.
Before the patch both the `dfsadmin -report` and the namenode webUI are
broken as described above. With the patch in place then both work as expected.
- All three datanode appear in the `dfsadmin -report` output and in the
namenode web UI with status "live".
- 10 minutes after the datanode pod was deleted the node appears as "dead"
in both `dfadmin -report` and the nodenode web UI. This is the default
behaviour expected for an offline datanode.
> HostSet.add() throws uncaught exception if DataNode hostname is not resolved
> ----------------------------------------------------------------------------
>
> Key: HDFS-17891
> URL: https://issues.apache.org/jira/browse/HDFS-17891
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.4.3
> Reporter: Jim Halfpenny
> Priority: Minor
> Labels: pull-request-available
>
> The HostSet.add() method that adds DataNode addresses to a HostSet checks to
> see if the hostname resolves. If this test fails then
> Preconditions.checkArgument() throws an uncaught IllegalArgumentException:
> {{ void add(InetSocketAddress addr) {}}
> {{ Preconditions.checkArgument(!addr.isUnresolved());}}
> {{ addrs.put(addr.getAddress(), addr.getPort());}}
> It would make more sense log the failure and return from the add() method
> rather than cause the creation of the HostSet to fail entirely. An example of
> when this issue manifests is when running `hdfs dfsadmin -report` on a
> cluster where one of the DataNode addresses cannot be resolved. The DataNode
> report fails with an IllegalArgumentException:
> {{stackable@simple-hdfs-namenode-default-0 /stackable/hadoop-3.4.0 $ hdfs
> dfsadmin -report}}
> {{Configured Capacity: 2933688373248 (2.67 TB)}}
> {{Present Capacity: 1299099013120 (1.18 TB)}}
> {{DFS Remaining: 1299098914816 (1.18 TB)}}
> {{DFS Used: 98304 (96 KB)}}
> {{DFS Used%: 0.00%}}
> {{Replicated Blocks:}}
> {{ Under replicated blocks: 0}}
> {{ Blocks with corrupt replicas: 0}}
> {{ Missing blocks: 0}}
> {{ Missing blocks (with replication factor 1): 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{Erasure Coded Block Groups:}}
> {{ Low redundancy block groups: 0}}
> {{ Block groups with corrupt internal blocks: 0}}
> {{ Missing block groups: 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{-------------------------------------------------}}
> {{report: java.lang.IllegalArgumentException}}
> The chance of a DataNode address not resolving is much more likely when HDFS
> is being run in Kubernetes; if a DataNode pod is removed the corresponding
> DNS entry will also be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
