[
https://issues.apache.org/jira/browse/HADOOP-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17942425#comment-17942425
]
ASF GitHub Bot commented on HADOOP-19518:
-----------------------------------------
anmolanmol1234 commented on code in PR #7564:
URL: https://github.com/apache/hadoop/pull/7564#discussion_r2037032596
##########
hadoop-tools/hadoop-azure/dev-support/testrun-scripts/configsupport.sh:
##########
@@ -0,0 +1,157 @@
+#!/usr/bin/env bash
+
+set -eo pipefail
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FILE=$1
+
+if [ ! -f "$FILE" ]; then
+ echo "Error: File '$FILE' not found. Exiting...."
+ exit 1
+fi
+
+if [[ "$FILE" != *.xml ]]; then
+ echo "The file provided is not an XML file. Exiting...."
+ exit 1
+fi
+
+OUTPUT_FILE="abfs-converted-config.xml"
+cp "$FILE" "$OUTPUT_FILE"
+
+contactTeamMsg="For any queries or support, kindly reach out to us at
'[email protected]'."
+endpoint=".dfs."
+printf "Select 'HNS' if you're migrating to ABFS driver for Hierarchical
Namespace enabled account,
+ or 'Non-HNS' if you're migrating to ABFS driver for Non-Hierarchical
Namespace (FNS) account. \n"
+printf "WARNING: Please ensure the correct option is chosen as it will affect
the configuration changes made to the file. \n"
+printf "If you are unsure, follow the instructions below to check from Azure
Portal: \n"
+printf "* Go to the Azure Portal and navigate to your storage account. \n"
+printf "* In the left-hand menu, select 'Overview' section and look for
'Properties'. \n"
+printf "* Under 'Blob service', check if 'Hierarchical namespace' is enabled
or disabled. \n"
+echo "$contactTeamMsg"
+select namespaceType in "HNS" "NonHNS"
+do
+ case $namespaceType in
+ HNS)
+ xmlstarlet ed -L -i '//configuration/property[1]' -t elem -n
property -v '' \
+ -s '//configuration/property[1]' -t elem -n name -v
'fs.azure.account.hns.enabled' \
+ -s '//configuration/property[1]' -t elem -n value -v 'true'
"$OUTPUT_FILE"
+ break;
+ ;;
+ NonHNS)
+ endpoint=".blob."
+ break;
+ ;;
+ *)
+ echo "Invalid selection. Please try again. Exiting..."
+ exit 1;
+ ;;
+ esac
+done
+
+# Mapping for renaming configurations
+declare -A renameConfigsMap=(
+ ["autothrottling.enable"]="enable.autothrottling"
#fs.azure.autothrottling.enable to fs.azure.enable.autothrottling
+ ["rename.dir"]="rename.key" # fs.azure.atomic.rename.dir to
fs.azure.atomic.rename.key
+ ["block.blob.buffered.pread.disable"]="buffered.pread.disable"
#fs.azure.block.blob.buffered.pread.disable to fs.azure.buffered.pread.disable
+ ["fs.azure.sas"]="fs.azure.sas.fixed.token."
#fs.azure.sas.CONTAINER_NAME.ACCOUNT_NAME to
fs.azure.sas.fixed.token.CONTAINER_NAME.ACCOUNT_NAME
+ ["check.block.md5"]="enable.checksum.validation" #fs.azure.check.block.md5
to fs.azure.enable.checksum.validation
+)
+
+# Configs not supported in ABFS
+unsupportedConfigsList=(
+ "fs.azure.page.blob.dir"
+ "fs.azure.block.blob.with.compaction.dir"
+ "fs.azure.store.blob.md5"
+)
+
+# Configurations not required in ABFS Driver and can be removed
+obseleteConfigsList=(
Review Comment:
typo: obsoleteConfigsList
> ABFS: [FnsOverBlob] WASB to ABFS Migration Config Support Script
> ----------------------------------------------------------------
>
> Key: HADOOP-19518
> URL: https://issues.apache.org/jira/browse/HADOOP-19518
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.4.0, 3.4.1
> Reporter: Manika Joshi
> Assignee: Manika Joshi
> Priority: Major
> Labels: pull-request-available
>
> The legacy WASB driver has been deprecated and is no longer recommended for
> use. To support customer onboard for migration from WASB to ABFS driver,
> we've introduced a script to help with the configuration changes required for
> the same.
> The script requires the configuration file (in XML format) used for WASB and
> would generate configuration file required for ABFS driver respectively.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
