[
https://issues.apache.org/jira/browse/KAFKA-16424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Chen resolved KAFKA-16424.
-------------------------------
Fix Version/s: 3.8.0
Resolution: Fixed
> truncated logs will be left undeleted after alter dir completion
> ----------------------------------------------------------------
>
> Key: KAFKA-16424
> URL: https://issues.apache.org/jira/browse/KAFKA-16424
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 3.7.0
> Reporter: Luke Chen
> Assignee: PoAn Yang
> Priority: Major
> Fix For: 3.8.0
>
>
> When doing log dir movement, we'll create a temp future replica with the dir
> named: topic-partition.uniqueId-future, ex:
> t3-0.9af8e054dbe249cf9379a210ec449af8-future. After the log dir movement
> completed, we'll rename the future log dir to the normal log dir, in the
> above case, it'll be "t3" only.
> So, if there are some logs to be deleted during the log dir movement, we'll
> send for a scheduler to do the deletion later
> ([here|https://github.com/apache/kafka/blob/2d4abb85bf4a3afb1e3359a05786ab8f3fda127e/core/src/main/scala/kafka/log/LocalLog.scala#L926]).
> However, when the log dir movement completed, the future log is renamed, the
> async log deletion will fail with no file existed error:
>
> {code:java}
> [2024-03-26 17:35:10,809] INFO [LocalLog partition=t3-0,
> dir=/tmp/kraft-broker-logs] Deleting segment files LogSegment(baseOffset=0,
> size=0, lastModifiedTime=0, largestRecordTimestamp=-1) (kafka.log.LocalLog$)
> [2024-03-26 17:35:10,810] INFO Failed to delete log
> /tmp/kraft-broker-logs/t3-0.9af8e054dbe249cf9379a210ec449af8-future/00000000000000000000.log.deleted
> because it does not exist.
> (org.apache.kafka.storage.internals.log.LogSegment)
> [2024-03-26 17:35:10,811] INFO Failed to delete offset index
> /tmp/kraft-broker-logs/t3-0.9af8e054dbe249cf9379a210ec449af8-future/00000000000000000000.index.deleted
> because it does not exist.
> (org.apache.kafka.storage.internals.log.LogSegment)
> [2024-03-26 17:35:10,811] INFO Failed to delete time index
> /tmp/kraft-broker-logs/t3-0.9af8e054dbe249cf9379a210ec449af8-future/00000000000000000000.timeindex.deleted
> because it does not exist.
> (org.apache.kafka.storage.internals.log.LogSegment) {code}
> I think we could consider fall back to the normal log dir if the future log
> dir cannot find the files. That is, when the file cannot be found under
> "t3-0.9af8e054dbe249cf9379a210ec449af8-future" dir, then try to find "t3"
> folder, and delete the file. Because the file is already having the suffix
> with ".delete", it should be fine if we delete them.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
