[
https://issues.apache.org/jira/browse/HDFS-17844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18064110#comment-18064110
]
ASF GitHub Bot commented on HDFS-17844:
---------------------------------------
balodesecurity opened a new pull request, #8332:
URL: https://github.com/apache/hadoop/pull/8332
### Problem
When multiple NFS export paths are configured on the same HDFS namenode
(e.g., `/path1` and `/path2` both served by `hdfs://namenode:8020`), the
`DFSClientCache` constructor throws a `FileSystemException` with the message
"Namenode ID collision". This causes the NFS gateway to fail startup even
though the configuration is valid.
The root symptom was a flaky test (`testViewFsMultipleExportPoint`) that
failed with:
```
Namenode ID collision for path:/hdfs2 nnid:2130740544
uri being added:hdfs://localhost:34111/
existing uri:hdfs://localhost:34111/
```
### Root Cause
`DFSClientCache.prepareAddressMap()` uses `Nfs3Utils.getNamenodeId()` to map
each export path to a namenode ID, where the ID is derived from the namenode's
`InetSocketAddress.hashCode()`. When two export paths resolve to the **same
namenode** (same host and port), they produce the same namenode ID. The
existing code incorrectly treats any duplicate namenode ID as a collision and
throws an error, even when both paths point to the same namenode authority.
The check should only fail when two **different** namenode authorities
(different host:port) hash to the same ID — a true hash collision — not when
the same namenode is referenced by multiple export paths.
### Fix
Changed the collision check in `prepareAddressMap()` from "throw on any
duplicate namenodeId" to "throw only when the conflicting entries have
different URI authorities (host:port)". When the authorities match, the
duplicate is logged at DEBUG level and skipped safely.
### Testing
Added `testMultipleExportPointsSameNamenode()` to `TestDFSClientCache` which
configures two export paths (`/path1` and `/path2`) on the same HDFS namenode
and verifies the `DFSClientCache` constructor completes without throwing.
Before the fix this test would fail with `FileSystemException`.
JIRA: https://issues.apache.org/jira/browse/HDFS-17844
> Flaky test: testViewFsMultipleExportPoint may encounter race condition which
> leads to namenode ID collisions
> ------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17844
> URL: https://issues.apache.org/jira/browse/HDFS-17844
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode, test
> Reporter: Yakun Wang
> Priority: Major
> Labels: patch, pull-request-available
>
> There could be a race condition in namenode ID generation, causing ID
> collisions. However, the underlying collision detection logic is incorrect
> because it throws an error as long as the namenode ID exists, even if the URI
> is the same. Same namenode ID with same URI is safe. What needs to be
> detected is that different URIs resolve to the same namenode ID, which is a
> security problem.
> Error output:
> `[ERROR] Errors:
> [ERROR] TestExportsTable.testViewFsMultipleExportPoint:111 » FileSystem
> FS:viewfs, Namenode ID collision for path:/hdfs2 nnid:2130740544 uri being
> added:hdfs://localhost:34111/ existing uri:hdfs://localhost:34111/`
> Functions that are related to collision detection logic:
> # prepareAddressMap in
> hadoop/hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/DFSClientCache.java
> # getNamenodeId in
> hadoop/hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/Nfs3Utils.java
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
