[
https://issues.apache.org/jira/browse/HADOOP-15723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16621090#comment-16621090
]
Thomas Marquardt commented on HADOOP-15723:
-------------------------------------------
[~kowon2008], looks good, thank you for providing Ranger support. A few
comments:
Your test command needs to include "verify" and you only need to paste the
summary results. Also, the test execution command-line was modified in
-HADOOP-15714,- and in your case you only need to run abfs tests, so the
command and example results are as follows:
{code:java}
$ mvn -T 1C -Dparallel-tests=abfs -Dscale -DtestsThreadCount=8 clean verify
Tests run: 35, Failures: 0, Errors: 0, Skipped: 0
Tests run: 268, Failures: 0, Errors: 0, Skipped: 182
Tests run: 167, Failures: 0, Errors: 0, Skipped: 27
{code}
If possible, all config and validation should be performed in
AbfsConfiguration.java. The original idea was that all config would be
validated when the filesystem is initialized, but I see we have broken from
that, probably thru loose code review. There is also some logic in
AbfsConfiguration to use attribute based validation (e.g. see readBufferSize).
Anyhow, looks like your AdlAuthorizerFactory.create should be a function
similar to AbfsConfiguration.getTokenProvider and getStorageAccountKey.
I see you introduced a new namespace: org.apache.hadoop.fs.azurebfs.auth. We
already have org.apache.hadoop.fs.azurebfs.oauth2 and
org.apache.hadoop.fs.azurebfs.security. I would recommend using
org.apache.hadoop.fs.azurebfs.security, although admittedly I am not sure why
we have a separate namespace for oauth2.
Currently the "extensions" are in org.apache.hadoop.fs.azurebfs.extensions, so
all the interfaces that can be implemented by providers outside of HADOOP are
defined in there. I think your AdlAuthorizer interface belongs there too.
The current file naming pattern involves prefixing files with "Abfs". I don't
necessarily agree with the decision to use this prefix, but since that is what
was done, I would recommend that you also use that prefix instead of "Adl".
> ABFS: Ranger Support
> --------------------
>
> Key: HADOOP-15723
> URL: https://issues.apache.org/jira/browse/HADOOP-15723
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: HADOOP-15407
> Reporter: Thomas Marquardt
> Assignee: Da Zhou
> Priority: Major
> Attachments: HADOOP-15273-HADOOP-15407-001.patch,
> HADOOP-15273-HADOOP-15407-002.patch, HADOOP-15273-HADOOP-15407-003.patch
>
>
> Add support for Ranger
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
