[
https://issues.apache.org/jira/browse/HDFS-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068552#comment-18068552
]
Anuj Modi commented on HDFS-17898:
----------------------------------
Hi [~ppothnis]
As per the Azure documentation, soft-delete feature is indeed supported if
requests are coming via BLOB endpoint. Soft-delete is not supported over DFS
Endpoint,
In your case if you look at the error message shared above the endpoint being
hit is DFS hence the error.
This seems like a case of mis-configuration. In the changes that you have
mentioned along with changing schema you have also changed endpoint from blob
to dfs which is causing the issue.
I would recommend following the [WASB to ABFS Migration
guide|[hadoop/hadoop-tools/hadoop-azure/src/site/markdown/wasbToAbfsMigration.md
at trunk ·
apache/hadoop|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/src/site/markdown/wasbToAbfsMigration.md]]
and run the [Migration
Script|[hadoop/hadoop-tools/hadoop-azure/dev-support/testrun-scripts/configsupport.sh
at trunk ·
apache/hadoop|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-azure/dev-support/testrun-scripts/configsupport.sh]]
to generate the valid set of configurations for ABFS Driver.
If you prefer manually changing the urls then make sure to use
.blob.windows.core.net only with FNS accounts for all abfs/abfss urls.
After fixing this Soft-delete error should not come. IN case you face any other
issue please share the stack trace, we should be able to help
> ABFS migration fails on FNS blob storage with Soft Delete enabled
> ------------------------------------------------------------------
>
> Key: HDFS-17898
> URL: https://issues.apache.org/jira/browse/HDFS-17898
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.4.2, 3.4.3
> Environment: *Environment*
> Hadoop version: _*3.4.2 and 3.4.3*_
> Storage SDK - Tried with both *6.x* and *12.X*
> Storage type: Azure Blob Storage (FNS - {*}FlatNameSpace{*})
> Storage account features enabled: FNS and Soft Delete
> Migration path: *WASB → ABFS (using abfss:// scheme)*
> Reporter: Pavan Pothnis
> Priority: Blocker
>
> When migrating from WASB to ABFS on a FlatNameSpace (FNS) Azure Blob Storage
> account that has both FNS and Soft Delete features enabled, the migration
> fails with an incompatibility error.
> *Environment*
> Hadoop version: _*3.4.2 and 3.4.3*_
> Storage SDK - Tried with both *6.x* and *12.X*
> Storage type: Azure Blob Storage (FNS - {*}FlatNameSpace{*})
> Storage account features enabled: FNS and Soft Delete
> Migration path: *WASB → ABFS (using abfss:// scheme)*
> *Issue Description*
> Following the WASB to ABFS migration guide (as documented in wasb.md), users
> are instructed to simply change URIs from WASB scheme to ABFS scheme.
> However, when the underlying Azure Blob Storage account has Soft Delete
> enabled, the ABFS driver encounters a critical error that prevents data
> access.
> *Steps to Reproduce*
> Create an Azure Blob Storage account (GPv2) with FlatNameSpace enabled
> Enable Soft Delete feature on the blob storage account
> Upgrade Hadoop from WASB to ABFS (3.4.2 )
> Change URIs from wasb:// to abfss://
> Attempt to read/parse Avro files or any data from the storage
> *Error Message*
> java.lang.RuntimeException: java.lang.RuntimeException: The file or partition
> directory
> [{*}abfss{*}://{*}azureblobv3-cctf{*}@{*}adapterqablob{*}.{*}dfs{*}.core.windows.net/alldatatype.avro]
> is not valid. The parser encountered the following error while parsing the
> content: [org.apache.hadoop.fs.FileAlreadyExistsException: Operation failed:
> "This endpoint does not support BlobStorageEvents or SoftDelete. Please
> disable these account features if you would like to use this endpoint.", 409,
> HEAD,
> https://adapterqablob.{*}dfs{*}.core.windows.net/azureblobv3-cctf/alldatatype.avro?upn=false&action=getStatus&timeout=90s,
> rId: 0b404e0b-301f-00a7-4c18-bc227f000000]. Select a valid [Avro] file or
> partition directory.
> Changes done as part of WABS to ABFS Migration
> *From -*
>
> String fileName = "wasbs://" +
> connection.getContainerClient().getBlobContainerName() + "@" +
> connection.getAccountName() + ".blob." +connection.getEndpointSuffix()+ "/";
>
> *To -*
>
> String fileName = "abfss://" +
> connection.getContainerClient().getBlobContainerName() + "@" +
> connection.getAccountName() + ".dfs." +connection.getEndpointSuffix()+ "/";
> *From -*
>
> String wasbFileSystemUrl = "wasbs://" + containerName + "@" +
> connection.getAccountName() +".blob."+connection.getEndpointSuffix()+"/";
>
> *To -*
>
> String wasbFileSystemUrl = "abfss://" + containerName + "@" +
> connection.getAccountName() +".dfs."+connection.getEndpointSuffix()+"/";
>
> *Configuration - Shared Key Authentication*
>
> if(authenticationType.equals(AzureStorageAuthorization.SHARED_KEY_AUTHORIZATION.getValue()))
> {
> *From -*
> String accountKey = "fs.azure.account.key." + connection.getAccountName() +
> ".blob." + endpointSuffix;
> *To -*
> **
> String accountKey = "fs.azure.account.key." + connection.getAccountName() +
> ".dfs." + endpointSuffix;
>
> configMap.put(accountKey, connection.getAccountKey());
>
> }
> *Configuration - Shared Access Signature*
>
> if(authenticationType.equals(AzureStorageAuthorization.SHARED_ACCESS_SIGNATURES.getValue()))
> {
> *From -*
> String containerName = StringUtils.isNotBlank(containerNameOverride) ?
> containerNameOverride : connection.getContainer().getName();
> String sharedaccesssignature = "fs.azure.sas." + containerName +"." +
> connection.getAccountName() + ".blob."+ endpointSuffix;
> configMap.put(sharedaccesssignature, connection.getSharedAccessSignature());
> *To -*
> configMap.put("fs.azure.account.auth.type." + connection.getAccountName() +
> ".dfs." + endpointSuffix, "SAS");
> configMap.put("fs.azure.sas.fixed.token." + connection.getAccountName() +
> ".dfs." + endpointSuffix, connection.getSharedAccessSignature());
>
> }
> All the changes made in To section works fine as long as the Blob account is
> not configured with Soft Delete / Event and fails when enabled, causing out
> of box failure for all account with those settings and these are extensively
> used settings.
>
> *Current Workaround*
> The issue only resolves after manually disabling Soft Delete on the Azure
> Blob Storage account. This workaround defeats the purpose of having this
> protective feature enabled and is not viable for production environments.
> *Expected Behavior*
> ABFS driver should support Soft Delete feature on FNS blob storage accounts,
> or the migration documentation should clearly document this incompatibility
> as a prerequisite check before migration.
> *Impact*
> Out-of-box failure for users migrating from WASB to ABFS on FNS storage with
> Soft Delete enabled
> Requires manual intervention and disabling of security features
> Migration path documented in wasb.md does not account for this limitation
> *Potential Solutions*
> Add support for Soft Delete in ABFS driver when using FNS blob storage
> Update WASB to ABFS migration guide to document this incompatibility and
> require Soft Delete to be disabled before migration
> Provide configuration option in ABFS to work with Soft Delete enabled
> accounts.
> Mention minimum required Hadoop Version for FNS Blob Storage with Soft delete
> migration from WASBS to ABFSS.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
