[
https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=592722&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-592722
]
ASF GitHub Bot logged work on HADOOP-17682:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 04/May/21 13:02
Start Date: 04/May/21 13:02
Worklog Time Spent: 10m
Work Description: vinaysbadami commented on a change in pull request
#2975:
URL: https://github.com/apache/hadoop/pull/2975#discussion_r625759702
##########
File path:
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##########
@@ -647,26 +650,38 @@ public void createDirectory(final Path path, final
FsPermission permission, fina
public AbfsInputStream openFileForRead(final Path path,
final FileSystem.Statistics statistics)
throws AzureBlobFileSystemException {
- return openFileForRead(path, Optional.empty(), statistics);
+ return openFileForRead(path, new OpenFileParameters(), statistics);
}
public AbfsInputStream openFileForRead(final Path path,
- final Optional<Configuration> options,
+ final OpenFileParameters parameters,
final FileSystem.Statistics statistics)
throws AzureBlobFileSystemException {
try (AbfsPerfInfo perfInfo = startTracking("openFileForRead",
"getPathStatus")) {
LOG.debug("openFileForRead filesystem: {} path: {}",
client.getFileSystem(),
path);
+ String resourceType;
+ long contentLength;
+ String eTag;
String relativePath = getRelativePath(path);
+ Configuration options = null;
- final AbfsRestOperation op = client.getPathStatus(relativePath, false);
- perfInfo.registerResult(op.getResult());
+ try {
+ options = parameters.getOptions();
+ FileStatus fileStatus = parameters.getStatus();
Review comment:
fileStatus can be null. If null should do line 678, 679. Then u will
have a valid filestatus if the getPathStatus call did not fail.
U can then set resourcetype, contentlenght and etgag from the filestatus.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 592722)
Time Spent: 20m (was: 10m)
> ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters
> ------------------------------------------------------------------------------
>
> Key: HADOOP-17682
> URL: https://issues.apache.org/jira/browse/HADOOP-17682
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Reporter: Sumangala Patki
> Assignee: Sumangala Patki
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ABFS open methods require certain information (contentLength, eTag, etc) to
> be sent to Store to open the file at the given path. This information is
> retrieved via a GetFileStatus request to backend, resulting in a total of two
> Store calls to carry out the open function.
> However, client applications may often have access to the FileStatus prior to
> invoking the open API. Providing this FileStatus to the driver through the
> OpenFileParameters argument of openFileWithOptions() can avoid the additional
> call to Store for FileStatus.
> This PR adds handling for the FileStatus instance (if any) provided via the
> OpenFileParameters argument.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
