[
https://issues.apache.org/jira/browse/HADOOP-17802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17380545#comment-17380545
]
Steve Loughran commented on HADOOP-17802:
-----------------------------------------
{code}
bin/hadoop fs -getfattr -d s3a://stevel-london/user/stevel/target/test/data
# file: s3a://stevel-london/user/stevel/target/test/data
2021-07-14 13:10:53,783 [main] DEBUG shell.Command
(Command.java:displayError(464)) - getfattr failure
java.io.FileNotFoundException: getObjectMetadata on
user/stevel/target/test/data/:
com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon
S3; Status Code: 404; Error Code: 404 Not Found; Request ID: MJ7KHEZ2R82NT179;
S3 Extended Request ID:
jWOXiMtgA8hmuVvwg4daFEwWnG0sTUa9g5+5nJ24GTlKTn1QDBSSnoXDylM6HYisdKjAI7c6nfE=;
Proxy: null), S3 Extended Request ID:
jWOXiMtgA8hmuVvwg4daFEwWnG0sTUa9g5+5nJ24GTlKTn1QDBSSnoXDylM6HYisdKjAI7c6nfE=:404
Not Found
at
org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:261)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:119)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$HeaderProcessingCallbacksImpl.getObjectMetadata(S3AFileSystem.java:4774)
at
org.apache.hadoop.fs.s3a.impl.HeaderProcessing.lambda$retrieveHeaders$1(HeaderProcessing.java:287)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:499)
at
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:444)
at
org.apache.hadoop.fs.s3a.impl.HeaderProcessing.retrieveHeaders(HeaderProcessing.java:286)
at
org.apache.hadoop.fs.s3a.impl.HeaderProcessing.getXAttrs(HeaderProcessing.java:405)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getXAttrs(S3AFileSystem.java:4727)
at
org.apache.hadoop.fs.shell.XAttrCommands$GetfattrCommand.processPath(XAttrCommands.java:107)
at
org.apache.hadoop.fs.shell.Command.processPathInternal(Command.java:370)
at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:333)
at
org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:306)
at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:288)
at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:272)
at
org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:121)
at org.apache.hadoop.fs.shell.Command.run(Command.java:179)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:327)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:95)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:390)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Not Found
(Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID:
MJ7KHEZ2R82NT179; S3 Extended Request ID:
jWOXiMtgA8hmuVvwg4daFEwWnG0sTUa9g5+5nJ24GTlKTn1QDBSSnoXDylM6HYisdKjAI7c6nfE=;
Proxy: null), S3 Extended Request ID:
jWOXiMtgA8hmuVvwg4daFEwWnG0sTUa9g5+5nJ24GTlKTn1QDBSSnoXDylM6HYisdKjAI7c6nfE=
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1819)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1403)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1372)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1145)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:802)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:770)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:744)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:704)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:686)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:550)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:530)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5437)
at
com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5384)
at
com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1367)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.lambda$getObjectMetadata$10(S3AFileSystem.java:2462)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:414)
at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:377)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2450)
at
org.apache.hadoop.fs.s3a.S3AFileSystem.getObjectMetadata(S3AFileSystem.java:2430)
at
org.apache.hadoop.fs.s3a.S3AFileSystem$HeaderProcessingCallbacksImpl.lambda$getObjectMetadata$0(S3AFileSystem.java:4775)
at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:117)
... 19 more
getfattr: getObjectMetadata on user/stevel/target/test/data/:
com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon
S3; Status Code: 404; Error Code: 404 Not Found; Request ID: MJ7KHEZ2R82NT179;
S3 Extended Request ID:
jWOXiMtgA8hmuVvwg4daFEwWnG0sTUa9g5+5nJ24GTlKTn1QDBSSnoXDylM6HYisdKjAI7c6nfE=;
Proxy: null), S3 Extended Request ID:
jWOXiMtgA8hmuVvwg4daFEwWnG0sTUa9g5+5nJ24GTlKTn1QDBSSnoXDylM6HYisdKjAI7c6nfE=:404
Not Found
{code}
> S3A Xattr/getXAttr to handle directories without markers
> --------------------------------------------------------
>
> Key: HADOOP-17802
> URL: https://issues.apache.org/jira/browse/HADOOP-17802
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.1
> Reporter: Steve Loughran
> Priority: Minor
>
> The XAttr support in S3 raises FNFE if given a directory as a path when that
> directory has children but no marker.
> It should somehow downgrade (e.g. FNFE -> probe for a dir, and if found
> return an empty set of headers)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
