[
https://issues.apache.org/jira/browse/HDFS-17947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18099924#comment-18099924
]
ASF GitHub Bot commented on HDFS-17947:
---------------------------------------
ayushtkn commented on code in PR #8638:
URL: https://github.com/apache/hadoop/pull/8638#discussion_r3671079495
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml:
##########
@@ -650,7 +650,12 @@
<property>
<name>dfs.block.access.token.lifetime</name>
<value>600</value>
- <description>The lifetime of access tokens in minutes.</description>
+ <description>The lifetime of access tokens in minutes. Must be greater than
+ zero. A non-positive value makes every block access token expire at the
+ instant it is created, which causes DataNodes to reject them and all write
+ pipelines to fail. When block access tokens are enabled, the NameNode
+ rejects a non-positive value and will neither format nor start.
+ </description>
Review Comment:
no need to change the description here
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:
##########
@@ -722,8 +722,17 @@ private static BlockTokenSecretManager
createBlockTokenSecretManager(
DFSConfigKeys.DFS_BLOCK_ACCESS_KEY_UPDATE_INTERVAL_KEY,
DFSConfigKeys.DFS_BLOCK_ACCESS_KEY_UPDATE_INTERVAL_DEFAULT);
final long lifetimeMin = conf.getLong(
- DFSConfigKeys.DFS_BLOCK_ACCESS_TOKEN_LIFETIME_KEY,
+ DFSConfigKeys.DFS_BLOCK_ACCESS_TOKEN_LIFETIME_KEY,
DFSConfigKeys.DFS_BLOCK_ACCESS_TOKEN_LIFETIME_DEFAULT);
+ if (lifetimeMin <= 0) {
+ throw new HadoopIllegalArgumentException(
+ DFSConfigKeys.DFS_BLOCK_ACCESS_TOKEN_LIFETIME_KEY + " = "
+ + lifetimeMin + " is invalid. It must be a positive number of"
+ + " minutes. A non-positive lifetime makes every block access"
+ + " token expire at the instant it is created, so DataNodes"
+ + " reject all of them and no write pipeline can be"
+ + " established.");
+ }
Review Comment:
This is too much information, just telling it is invalid and should be +ve
is more than enoguh
##########
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/security/token/block/TestBlockTokenZeroLifetime.java:
##########
@@ -0,0 +1,185 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hdfs.security.token.block;
+
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.io.ByteArrayInputStream;
+import java.io.DataInputStream;
+import java.io.IOException;
+import java.util.EnumSet;
+
+import org.apache.hadoop.fs.StorageType;
+import org.apache.hadoop.hdfs.protocol.ExtendedBlock;
+import org.apache.hadoop.security.token.SecretManager.InvalidToken;
+import org.apache.hadoop.security.token.Token;
+import org.apache.hadoop.util.Time;
+import org.junit.jupiter.api.Test;
+
+/**
+ * Tests the behaviour of {@link BlockTokenSecretManager} when it is configured
+ * with a token lifetime of zero, which is what
+ * {@code dfs.block.access.token.lifetime = 0} produces (HDFS-17947).
+ *
+ * A zero lifetime makes every token expire at the instant it is minted, so it
+ * is rejected by the very next verification. These tests pin that behaviour
+ * down at the secret-manager level, without needing a cluster.
+ */
+public class TestBlockTokenZeroLifetime {
Review Comment:
adjust somewhere in a related exisiting test rather than adding a new class
> dfs.block.access.token.lifetime=0 can cause DataStreamer
> createBlockOutputStream failure
> -----------------------------------------------------------------------------------------
>
> Key: HDFS-17947
> URL: https://issues.apache.org/jira/browse/HDFS-17947
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 3.4.3
> Reporter: jiang he
> Priority: Major
> Labels: pull-request-available
>
> When block access tokens are enabled and dfs.block.access.token.lifetime is
> set to 0, HDFS write operations fail with a DataStreamer error.
> Reproduced with Hadoop 3.4.3.
> The same test passes when dfs.block.access.token.lifetime is set to a normal
> positive value.
> Reproduction config
> <configuration>
> <property>
> <name>dfs.replication</name>
> <value>1</value>
> </property>
> <property>
> <name>dfs.datanode.du.reserved.calculator</name>
>
> <value>org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.ReservedSpaceCalculator$ReservedSpaceCalculatorAbsolu
> te</value>
> </property>
> <property>
> <name>dfs.block.access.token.lifetime</name>
> <value>0</value>
> </property>
> <property>
> <name>dfs.namenode.upgrade.domain.factor</name>
> <value>${dfs.replication}</value>
> </property>
> <property>
> <name>dfs.ha.zkfc.nn.http.timeout.ms</name>
> <value>20000</value>
> </property>
> <property>
> <name>dfs.namenode.reencrypt.throttle.limit.updater.ratio</name>
> <value>1.0</value>
> </property>
> <property>
> <name>dfs.namenode.redundancy.considerLoadByVolume</name>
> <value>false</value>
> </property>
> <property>
> <name>dfs.datanode.du.reserved</name>
> <value>0</value>
> </property>
> <property>
> <name>dfs.datanode.du.reserved.pct</name>
> <value>0</value>
> </property>
> <property>
> <name>dfs.block.access.token.enable</name>
> <value>True</value>
> </property>
> <property>
> <name>dfs.replication</name>
> <value>3</value>
> </property>
> <property>
> <name>dfs.namenode.reencrypt.throttle.limit.handler.ratio</name>
> <value>1.0</value>
> </property>
> </configuration>
> Observed
> WARN hdfs.DataStreamer:
> Exception in createBlockOutputStream blk_1073741825_1001
> The API test fails while writing to HDFS.
> Full observed wrapper output:
> API operation exception and shut done hdfs!
> Exception in thread "main" java.lang.RuntimeException:
> API request Exception: Operation exception[info_excetion]
> 2026-06-26 15:59:40,444 WARN hdfs.DataStreamer:
> Exception in createBlockOutputStream blk_1073741825_1001
> Expected
> Either dfs.block.access.token.lifetime=0 should be rejected with a clear
> configuration validation error, or HDFS should handle this value without
> causing DataStreamer write failure.
> Control test
> dfs.block.access.token.enable=true
> dfs.block.access.token.lifetime=600
> passes successfully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
