[
https://issues.apache.org/jira/browse/HADOOP-9377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596124#comment-13596124
]
David Capwell commented on HADOOP-9377:
---------------------------------------
Seeing how you cant set the home dir, nor should it be mutable via another
service (since it would have been set for the session), cacheing the home dir
would be safe to do. Even if the home dir is cached, most of the ftp logic
still would require the current state of WD (setWD is stubbed out, but might be
implemented in the future). If you go with the solution in the patch to pass
the WD that was used during the list operation, you would future prof this
call. Example, if you list a dir with 3k files in it and another thread
updates the WD, this would cause list to now use the wrong WD (locking would
solve this) for part of the files; passing the known WD would be safer.
> FTPFileSystem.listStatus() runs very slow, due to inappropriate call of
> filePath.makeQualified
> ----------------------------------------------------------------------------------------------
>
> Key: HADOOP-9377
> URL: https://issues.apache.org/jira/browse/HADOOP-9377
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.3-alpha
> Reporter: James Yu
> Attachments: HADOOP-9377.diff
>
>
> FTPFileSystem.listStatus() calls
> getFileStatus(ftpFiles[i], absolute) calls
> new FileStatus(....) calls
> filePath.makeQualified(...) calls
> fs.getWorkingDirectory() calls
> getHomeDirectory()
> which creates new FTP connection every time, to get the workdingDirectory.
> this caused the FTPFileSystem.listStatus() takes long time to run (on average
> 3-6 seconds per file in my test).
> I attach a suggestion of fix in FTPFileSystem.java, only 4 lines of change.
> after the fix, there's no slowness issue anymore.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
