[
https://issues.apache.org/jira/browse/HADOOP-14439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110842#comment-16110842
]
Vinayakumar B edited comment on HADOOP-14439 at 8/2/17 1:00 PM:
----------------------------------------------------------------
Re-adding the secret to S3X urI in the same secret encoded format.
Following are the highlights of changes:
1. Builds the FSUri containing only schema and authority part.
2. Strips down secret part if {{user}} was not provided in the {{userinfo}}
section.
3. Encodes the secret part, even if original passed uri was not containing
encoded secret.
So, direct comparison of fs uri and provided uri still might fail in above
cases.
Please review.
Ran TestS3xLoginHelper test, didnt run any Integration tests, as dont have any
accessible s3 environments.
was (Author: vinayrpet):
Re-adding the secret to S3X urI in the same secret encoded format.
Following are the highlights of changes:
1. Builds the FSUri containing only schema and authority part.
2. Strips down secret part if {{user}} was not provided in the {{userinfo}}
section.
3. Encodes the secret part, even if original passed uri was not containing
encoded secret.
So, direct comparison of fs uri and provided uri still might fail in above
cases.
Please review.
> regression: secret stripping from S3x URIs breaks some downstream code
> ----------------------------------------------------------------------
>
> Key: HADOOP-14439
> URL: https://issues.apache.org/jira/browse/HADOOP-14439
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 2.8.0
> Environment: Spark 2.1
> Reporter: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-14439-01.patch
>
>
> Surfaced in SPARK-20799
> Spark is listing the contents of a path with getFileStatus(path), then
> looking up the path value doing a lookup of the contents.
> Apparently the lookup is failing to find files if you have a secret in the
> key, {{s3a://key:secret@bucket/path}}.
> Presumably this is because the stripped values aren't matching.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
