[
https://issues.apache.org/jira/browse/HDFS-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18078804#comment-18078804
]
ASF GitHub Bot commented on HDFS-17891:
---------------------------------------
Jimvin commented on PR #8341:
URL: https://github.com/apache/hadoop/pull/8341#issuecomment-4392878527
I've added tests for HostSet to the PR. There were none there to begin with
but adding them now means the skipping unresolved hosts is tested as well as
the expected behaviour.
> HostSet.add() throws uncaught exception if DataNode hostname is not resolved
> ----------------------------------------------------------------------------
>
> Key: HDFS-17891
> URL: https://issues.apache.org/jira/browse/HDFS-17891
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.4.3
> Reporter: Jim Halfpenny
> Assignee: Jim Halfpenny
> Priority: Minor
> Labels: pull-request-available
>
> The HostSet.add() method that adds DataNode addresses to a HostSet checks to
> see if the hostname resolves. If this test fails then
> Preconditions.checkArgument() throws an uncaught IllegalArgumentException:
> {{ void add(InetSocketAddress addr) {}}
> {{ Preconditions.checkArgument(!addr.isUnresolved());}}
> {{ addrs.put(addr.getAddress(), addr.getPort());}}
> It would make more sense log the failure and return from the add() method
> rather than cause the creation of the HostSet to fail entirely. An example of
> when this issue manifests is when running `hdfs dfsadmin -report` on a
> cluster where one of the DataNode addresses cannot be resolved. The DataNode
> report fails with an IllegalArgumentException:
> {{stackable@simple-hdfs-namenode-default-0 /stackable/hadoop-3.4.0 $ hdfs
> dfsadmin -report}}
> {{Configured Capacity: 2933688373248 (2.67 TB)}}
> {{Present Capacity: 1299099013120 (1.18 TB)}}
> {{DFS Remaining: 1299098914816 (1.18 TB)}}
> {{DFS Used: 98304 (96 KB)}}
> {{DFS Used%: 0.00%}}
> {{Replicated Blocks:}}
> {{ Under replicated blocks: 0}}
> {{ Blocks with corrupt replicas: 0}}
> {{ Missing blocks: 0}}
> {{ Missing blocks (with replication factor 1): 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{Erasure Coded Block Groups:}}
> {{ Low redundancy block groups: 0}}
> {{ Block groups with corrupt internal blocks: 0}}
> {{ Missing block groups: 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{-------------------------------------------------}}
> {{report: java.lang.IllegalArgumentException}}
> The chance of a DataNode address not resolving is much more likely when HDFS
> is being run in Kubernetes; if a DataNode pod is removed the corresponding
> DNS entry will also be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
