[
https://issues.apache.org/jira/browse/HADOOP-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596310#comment-13596310
]
Andrew Wang commented on HADOOP-9377:
-------------------------------------
Reading in more, the semantics for this filesystem are rather loose, since we
create a new FTPClient within each of the public FileSystem methods. Thus,
there's no persistent concept of a working directory, and everything relies on
absolute paths. Seems like FTPFileSystem could use some love beyond this JIRA.
Since the current working directory can be gotten from
{{FTPClient#printWorkingDirectory}}, I think we could simply use that in
{{getFileStatus}} without passing in a separate {{workDir}} parameter.
[~sureshms] I think we can cache the home dir, we just need to set it in
{{initialize()}}. I don't think homedirs don't change often, if at all, so
pretty safe.
[[email protected]] Poking in the test directory, I see no tests.
> FTPFileSystem.listStatus() runs very slow, due to inappropriate call of
> filePath.makeQualified
> ----------------------------------------------------------------------------------------------
>
> Key: HADOOP-9377
> URL: https://issues.apache.org/jira/browse/HADOOP-9377
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.3-alpha
> Reporter: James Yu
> Attachments: HADOOP-9377.diff
>
>
> FTPFileSystem.listStatus() calls
> getFileStatus(ftpFiles[i], absolute) calls
> new FileStatus(....) calls
> filePath.makeQualified(...) calls
> fs.getWorkingDirectory() calls
> getHomeDirectory()
> which creates new FTP connection every time, to get the workdingDirectory.
> this caused the FTPFileSystem.listStatus() takes long time to run (on average
> 3-6 seconds per file in my test).
> I attach a suggestion of fix in FTPFileSystem.java, only 4 lines of change.
> after the fix, there's no slowness issue anymore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
