[
https://issues.apache.org/jira/browse/HADOOP-13252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886967#comment-15886967
]
Mingliang Liu commented on HADOOP-13252:
----------------------------------------
{code}
294 /**
295 * Create an AWS credential provider.
296 * @param conf configuration
297 * @param credClass credential class
298 * @param uri URI of the FS
299 * @return the instantiated class
300 * @throws IOException on any instantiation failure.
301 */
302 static AWSCredentialsProvider createAWSCredentialProvider(
303 Configuration conf,
304 Class<?> credClass, URI uri) throws IOException {
305 AWSCredentialsProvider credentials;
306 String className = credClass.getName();
307 try {
308 LOG.debug("Credential provider class is {}", className);
309 try {
310 credentials =
311 (AWSCredentialsProvider) credClass.getDeclaredConstructor(
312 URI.class, Configuration.class).newInstance(uri, conf);
313 } catch (NoSuchMethodException | SecurityException e) {
314 credentials =
315 (AWSCredentialsProvider) credClass.getDeclaredConstructor()
316 .newInstance();
317 }
318 } catch (NoSuchMethodException | SecurityException e) {
319 throw new IOException(String.format("%s constructor exception. A
"
320 + "class specified in %s must provide an accessible
constructor "
321 + "accepting URI and Configuration, or an accessible default "
322 + "constructor.", className, AWS_CREDENTIALS_PROVIDER), e);
323 } catch (ReflectiveOperationException | IllegalArgumentException e)
{
324 throw new IOException(className + " instantiation exception.", e);
325 }
326 LOG.debug("Using {} for {}.", credentials, uri);
327 return credentials;
328 }
{code}
It looks like the URI parameter is not needed for our AWSCredentialProvider
constructors. Can I file a new JIRA to remove it, if this is true?
For user:pass in binding URIs, we have
{code}
279 S3xLoginHelper.Login creds = getAWSAccessKeys(binding, conf);
280 credentials.add(new BasicAWSCredentialsProvider(
281 creds.getUser(), creds.getPassword()));
{code}
This way, we only need configuration object (if necessary) for new
AWSCredentialProvider implementations. The benefit is that, if we create
AWSCredentialProvider list for DynamoDB, we don't have to pass down the
associated file system URI. This might be useful to S3Guard tools.
> Tune S3A provider plugin mechanism
> ----------------------------------
>
> Key: HADOOP-13252
> URL: https://issues.apache.org/jira/browse/HADOOP-13252
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Fix For: 2.8.0, 3.0.0-alpha1
>
> Attachments: HADOOP-13252-006.patch, HADOOP-13252-007.patch,
> HADOOP-13252-branch-2-001.patch, HADOOP-13252-branch-2-003.patch,
> HADOOP-13252-branch-2-004.patch, HADOOP-13252-branch-2-005.patch
>
>
> We've now got some fairly complex auth mechanisms going on: -hadoop config,
> KMS, env vars, "none". IF something isn't working, it's going to be a lot
> harder to debug.
> Review and tune the S3A provider point
> * add logging of what's going on in s3 auth to help debug problems
> * make a whole chain of logins expressible
> * allow the anonymous credentials to be included in the list
> * review and updated documents.
> I propose *carefully* adding some debug messages to identify which auth
> provider is doing the auth, so we can see if the env vars were kicking in,
> sysprops, etc.
> What we mustn't do is leak any secrets: this should be identifying whether
> properties and env vars are set, not what their values are. I don't believe
> that this will generate a security risk.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
