[
https://issues.apache.org/jira/browse/HADOOP-13345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15811874#comment-15811874
]
Steve Loughran commented on HADOOP-13345:
-----------------------------------------
-1 to anything working with user:pass in URIs. IT's wrong because it
contaminates the logs with secrets. With HADOOP-13336 you can define per-bucket
secrets in core-site.xml, or the command line with {{-D
fs.s3a.bucket.stevel-new.aws.secret.id=mykey}}; they'll get picked up in he FS
instance.
This means that the bucket URI will be needed for setup, but it doesn't have to
involve instantiating the FS, just:
{code}
bucket = new URI(fsName).getHost()
conf = propagateBucketOptions(origConf, bucket)
s3ClientFactoryClass = conf.getClass(
S3_CLIENT_FACTORY_IMPL, DEFAULT_S3_CLIENT_FACTORY_IMPL,
S3ClientFactory.class);
s3Client = ReflectionUtils.newInstance(s3ClientFactoryClass, conf)
.createS3Client(name, uri);
{code}
Thats all. But we will need that bucket name if there's any need to go near
bucket-related options
> S3Guard: Improved Consistency for S3A
> -------------------------------------
>
> Key: HADOOP-13345
> URL: https://issues.apache.org/jira/browse/HADOOP-13345
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-13345.prototype1.patch,
> S3C-ConsistentListingonS3-Design.pdf, S3GuardImprovedConsistencyforS3A.pdf,
> S3GuardImprovedConsistencyforS3AV2.pdf, s3c.001.patch
>
>
> This issue proposes S3Guard, a new feature of S3A, to provide an option for a
> stronger consistency model than what is currently offered. The solution
> coordinates with a strongly consistent external store to resolve
> inconsistencies caused by the S3 eventual consistency model.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
