[
https://issues.apache.org/jira/browse/HADOOP-13037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644926#comment-15644926
]
Chris Nauroth commented on HADOOP-13037:
----------------------------------------
[~chris.douglas] and [~vishwajeet.dusane], I advise setting
fs.contract.is-blobstore to false. This flag is now strictly informational and
does not control behavior of any tests. Since ADL is not a file system tree
mapped onto a flat object store, it makes sense to set it to false.
HADOOP-13502 introduced two new flags to control test behavior previously
controlled by fs.contract.is-blobstore. For ADL, I believe the correct match
to its semantics would be fs.contract.create-overwrites-directory=false and
fs.contract.create-visibility-delayed=false. The latter is certainly important
to support HBase expectations, and I know you want ADL to be able to support
HBase. Running the ADL subclass of {{AbstractContractCreateTest}} would
demonstrate if ADL is successfully implementing these semantics.
FYI, I am about one week away from having a viable build environment, so I'll
be delayed on testing the rebased patch that I mentioned. HADOOP-13687 likely
will get committed without moving ADL in the source tree. We can likely
accomplish that move within the scope of this JIRA instead.
> Azure Data Lake Client: Support Azure data lake as a file system in Hadoop
> --------------------------------------------------------------------------
>
> Key: HADOOP-13037
> URL: https://issues.apache.org/jira/browse/HADOOP-13037
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/azure, tools
> Reporter: Shrikant Naidu
> Assignee: Vishwajeet Dusane
> Fix For: 2.9.0
>
> Attachments: HADOOP-13037 Proposal.pdf, HADOOP-13037-001.patch,
> HADOOP-13037-002.patch, HADOOP-13037-003.patch, HADOOP-13037-004.patch,
> HADOOP-13037.005.patch
>
>
> The jira proposes an improvement over HADOOP-12666 to remove webhdfs
> dependencies from the ADL file system client and build out a standalone
> client. At a high level, this approach would extend the Hadoop file system
> class to provide an implementation for accessing Azure Data Lake. The scheme
> used for accessing the file system will continue to be
> adl://<accountname>.azuredatalake.net/path/to/file.
> The Azure Data Lake Cloud Store will continue to provide a webHDFS rest
> interface. The client will access the ADLS store using WebHDFS Rest APIs
> provided by the ADLS store.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
