[
https://issues.apache.org/jira/browse/HDFS-17899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18068661#comment-18068661
]
ASF GitHub Bot commented on HDFS-17899:
---------------------------------------
JHSUYU opened a new pull request, #8383:
URL: https://github.com/apache/hadoop/pull/8383
JIRA: HDFS-17899
## Summary
This is a follow-up of HDFS-17897.
HDFS-17897 fixed `InvalidEncryptionKeyException` handling in
DFSClient read/write and striped file checksum paths. However, three other
code paths that establish SASL-encrypted connections still lack this
handling:
1. `Dispatcher.PendingMove.dispatch()` — Balancer block moves
2. `BlockDispatcher.moveBlock()` — SPS block moves
3. `DataNode.DataTransfer.run()` — DataNode block replication
When `dfs.encrypt.data.transfer=true` and block keys rotate, these paths
fail
with `InvalidEncryptionKeyException` and the stale key stays cached,
causing
all subsequent transfers to fail until process restart.
**Fix:** Add the same retry pattern to all three paths — catch the
exception,
clear the cached encryption key via a new `clearDataEncryptionKey()`
default
method on `DataEncryptionKeyFactory`, and retry once with a fresh key.
## Test
- [x] `TestKeyManager#testClearDataEncryptionKey` — verifies
`KeyManager.clearDataEncryptionKey()` clears cached key
- [x] `TestDispatcherEncryptionKey#testClearEncryptionKeyOnRetry` —
verifies
Balancer Dispatcher retry
- [x] `TestBlockDispatcher#testClearEncryptionKeyOnRetry` — verifies SPS
BlockDispatcher retry
- [x] `TestDataTransferEncryptionKey#testClearEncryptionKeyOnRetry` —
verifies
DataNode DataTransfer retry
> Handle InvalidEncryptionKeyException in Balancer Dispatcher, SPS
> BlockDispatcher and DataNode DataTransfer
> -----------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17899
> URL: https://issues.apache.org/jira/browse/HDFS-17899
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: balancer & mover, datanode, encryption
> Affects Versions: 3.4.1
> Reporter: ZhenyuLi
> Priority: Major
>
> This is a follow-up of HDFS-17897.
> HDFS-17897 fixed {{InvalidEncryptionKeyException}} handling in DFSClient
> read/write and striped file checksum paths. However, three other
> code paths that establish SASL-encrypted connections still lack this
> handling:
> {{Dispatcher.PendingMove.dispatch()}} — Balancer block moves
> {{BlockDispatcher.moveBlock()}} — SPS block moves
> {{DataNode.DataTransfer.run()}} — DataNode block replication
> When {{dfs.encrypt.data.transfer=true}} and block keys rotate, these paths
> are possible to fail with {{InvalidEncryptionKeyException}} and the stale key
> stays cached, causing all subsequent transfers to fail until process restart.
> Fix: Add the same retry pattern to all three paths — catch the exception,
> clear the cached encryption key via a new {{clearDataEncryptionKey()}}
> default
> method on {{DataEncryptionKeyFactory}}, and retry once with a fresh key.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
