[
https://issues.apache.org/jira/browse/HDFS-17891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18065447#comment-18065447
]
ASF GitHub Bot commented on HDFS-17891:
---------------------------------------
Jimvin opened a new pull request, #8341:
URL: https://github.com/apache/hadoop/pull/8341
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
This patch changes the hostname resolution check in HostSet.add() to skip
unresolved hosts rather than throwing an IllegalArgumentException. It fixes
`hdfs dfsadmin -report` where one or more datanode hostnames are unresolvable.
### For code changes:
- [x] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
### AI Tooling
AI tooling did not write this code.
> HostSet.add() throws uncaught exception if DataNode hostname is not resolved
> ----------------------------------------------------------------------------
>
> Key: HDFS-17891
> URL: https://issues.apache.org/jira/browse/HDFS-17891
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.4.3
> Reporter: Jim Halfpenny
> Priority: Minor
>
> The HostSet.add() method that adds DataNode addresses to a HostSet checks to
> see if the hostname resolves. If this test fails then
> Preconditions.checkArgument() throws an uncaught IllegalArgumentException:
> {{ void add(InetSocketAddress addr) {}}
> {{ Preconditions.checkArgument(!addr.isUnresolved());}}
> {{ addrs.put(addr.getAddress(), addr.getPort());}}
> It would make more sense log the failure and return from the add() method
> rather than cause the creation of the HostSet to fail entirely. An example of
> when this issue manifests is when running `hdfs dfsadmin -report` on a
> cluster where one of the DataNode addresses cannot be resolved. The DataNode
> report fails with an IllegalArgumentException:
> {{stackable@simple-hdfs-namenode-default-0 /stackable/hadoop-3.4.0 $ hdfs
> dfsadmin -report}}
> {{Configured Capacity: 2933688373248 (2.67 TB)}}
> {{Present Capacity: 1299099013120 (1.18 TB)}}
> {{DFS Remaining: 1299098914816 (1.18 TB)}}
> {{DFS Used: 98304 (96 KB)}}
> {{DFS Used%: 0.00%}}
> {{Replicated Blocks:}}
> {{ Under replicated blocks: 0}}
> {{ Blocks with corrupt replicas: 0}}
> {{ Missing blocks: 0}}
> {{ Missing blocks (with replication factor 1): 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{Erasure Coded Block Groups:}}
> {{ Low redundancy block groups: 0}}
> {{ Block groups with corrupt internal blocks: 0}}
> {{ Missing block groups: 0}}
> {{ Low redundancy blocks with highest priority to recover: 0}}
> {{ Pending deletion blocks: 0}}
> {{-------------------------------------------------}}
> {{report: java.lang.IllegalArgumentException}}
> The chance of a DataNode address not resolving is much more likely when HDFS
> is being run in Kubernetes; if a DataNode pod is removed the corresponding
> DNS entry will also be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
