[
https://issues.apache.org/jira/browse/HDFS-17960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18103871#comment-18103871
]
ASF GitHub Bot commented on HDFS-17960:
---------------------------------------
JHSUYU opened a new pull request, #8672:
URL: https://github.com/apache/hadoop/pull/8672
<!--
Thanks for sending a pull request!
1. If this is your first time, please read our contributor guidelines:
https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute
2. Make sure your PR title starts with JIRA issue id, e.g.,
'HADOOP-17799. Your PR title ...'.
-->
### Description of PR
This PR fixes [HDFS-17960](https://issues.apache.org/jira/browse/HDFS-17960).
When edit logs generated with ordered snapshot deletion enabled are replayed
with ordered deletion disabled, multiple delete records may refer to the same
snapshot. After the first record removes the snapshot
`DirectorySnapshottableFeature.removeSnapshot()` returns `null` for a later
duplicate record.
`SnapshotManager.deleteSnapshot()` previously ignored this return value and
unconditionally decremented `numSnapshots`. This caused the counter to differ
from the snapshots actually present in the namespace. A subsequent checkpoint
could fail while serializing the snapshot section because the number of
serialized snapshots did not match`numSnapshots`.
This change decrements `numSnapshots` only when `removeSnapshot()` actually
returns a removed snapshot.
### How was this patch tested?
This PR extends the existing ordered snapshot deletion restart test to
verify:
- The internal snapshot count matches the snapshot listing.
- `saveNamespace()` can successfully create a checkpoint.
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [x] Object storage: Not applicable; this change only affects HDFS
NameNode
snapshot handling.
- [x] No new dependencies are added by this change.
- [x] No LICENSE, LICENSE-binary, or NOTICE-binary updates are required.
### AI Tooling
If an AI tool was used:
- [x] Contains content generated by Codex
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
> Snapshot count becomes inconsistent when replaying ordered snapshot deletion
> edits
> -----------------------------------------------------------------------------------
>
> Key: HDFS-17960
> URL: https://issues.apache.org/jira/browse/HDFS-17960
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: snapshots
> Affects Versions: 3.5.0
> Reporter: ZhenyuLi
> Priority: Major
>
> When edit logs generated with ordered snapshot deletion enabled are replayed
> with dfs.namenode.snapshot.deletion.ordered=false,
> SnapshotManager.numSnapshots may be decremented even though no snapshot is
> removed.
> HDFS-15590 added a replay-tolerance path in
> DirectorySnapshottableFeature.removeSnapshot(). If the requested snapshot
> does not exist, ordered deletion is disabled, and the namespace image is
> still being loaded, removeSnapshot() returns null:
> {code:java}
> if (!snapshotManager.isSnapshotDeletionOrdered()
> && !snapshotManager.isImageLoaded()) {
> return null;
> }
> {code}
> However, SnapshotManager.deleteSnapshot() ignores the return value and
> unconditionally decrements numSnapshots:
> {code:java}
> srcRoot.removeSnapshot(reclaimContext, snapshotName, now, this);
> numSnapshots.getAndDecrement();
> {code}
> Therefore, a tolerated no-op edit changes numSnapshots without changing the
> authoritative snapshotsByNames lists.
> h3. Proposed fix
> Use the return value of removeSnapshot() and decrement numSnapshots only
> when
> a Snapshot was actually removed:
> {code:java}
> final Snapshot removed = srcRoot.removeSnapshot(
> reclaimContext, snapshotName, now, this);
> if (removed != null) {
> numSnapshots.decrementAndGet();
> }
> {code}
> The resulting snapshot count inconsistency causes subsequent checkpoint or
> saveNamespace operations to fail at Preconditions.checkState(i ==
> sm.getNumSnapshots()) in
> FSImageFormatPBSnapshot.Saver.serializeSnapshotSection().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
