[
https://issues.apache.org/jira/browse/HADOOP-9972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770496#comment-13770496
]
Binglin Chang commented on HADOOP-9972:
---------------------------------------
Regarding API, I think we should differentiate core API and extend/legacy API,
IMO, there should be 3 core API:
getFileStatus resolve symlink
getFileLinkStatus don't resolve symlink
readdir don't resolve symlink, just like current HDFS listStatus
These core API should be implemented in each FS
All other related APIs can be build based on core API and implemented in
FSContext/FileSystem once for all:
{code}
FS.listStatus(path):
readdir(path).map(s => if (s.isSymlink) getFileStatus ignore Exception else s)
FS.listStatus(path, PathOptions):
readdir(path).map(process PathOptions)
glob(pattern):
if pattern matches none, return pattern
else return matched paths
ignore all exceptions
globStatus(pattern):
glob(pattern).map(getFileStatus)
{code}
> new APIs for listStatus and globStatus to deal with symlinks
> ------------------------------------------------------------
>
> Key: HADOOP-9972
> URL: https://issues.apache.org/jira/browse/HADOOP-9972
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 2.1.1-beta
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
>
> Based on the discussion in HADOOP-9912, we need new APIs for FileSystem to
> deal with symlinks. The issue is that code has been written which is
> incompatible with the existence of things which are not files or directories.
> For example,
> there is a lot of code out there that looks at FileStatus#isFile, and
> if it returns false, assumes that what it is looking at is a
> directory. In the case of a symlink, this assumption is incorrect.
> It seems reasonable to make the default behavior of {{FileSystem#listStatus}}
> and {{FileSystem#globStatus}} be fully resolving symlinks, and ignoring
> dangling ones. This will prevent incompatibility with existing MR jobs and
> other HDFS users. We should also add new versions of listStatus and
> globStatus that allow new, symlink-aware code to deal with symlinks as
> symlinks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
