[
https://issues.apache.org/jira/browse/KAFKA-16463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-16463:
---------------------------------
Description:
Throughout the process of a ZK to KRaft migration, the operator has the choice
to revert back to ZK mode. Once this is done, there will be a copy of the
metadata log on each broker in the cluster.
In order to re-attempt the migration in the future, this metadata log needs to
be deleted. This can be pretty burdensome to the operator for large clusters,
especially since the log deletion must be done while the broker is offline.
To improve this, we can automatically delete any metadata log present during
startup of a ZK broker. In general, it is always safe to remove the metadata
log from a KRaft or migrating ZK broker. The main impact is that this will
delay the time it takes for the broker to be unfenced by the controller since
it has to re-replicate the log. In the case of hybrid mode ZK brokers, there
will be a delay in them receiving their first UpdateMetadataRequest from the
controller (for the same reason -- delay in getting unfenced).
The delayed startup should not affect the performance of the cluster, though it
would increase the overall time required to do a rolling restart of the
cluster.
Once a broker restarts as KRaft, we will stop doing this automatic deletion.
was:
Throughout the process of a ZK to KRaft migration, the operator has the choice
to revert back to ZK mode. Once this is done, there will be a copy of the
metadata log on each broker in the cluster.
In order to re-attempt the migration in the future, this metadata log needs to
be deleted. This can be pretty burdensome to the operator for large clusters.
To improve this, we can automatically delete any metadata log present during
startup of a ZK broker. This is safe to do because the ZK broker will just
re-replicate the metadata log from the active controller. Once a broker
restarts as KRaft, it will stop doing this automatic deletion.
> Automatically delete metadata log directory on ZK brokers
> ---------------------------------------------------------
>
> Key: KAFKA-16463
> URL: https://issues.apache.org/jira/browse/KAFKA-16463
> Project: Kafka
> Issue Type: Improvement
> Reporter: David Arthur
> Assignee: David Arthur
> Priority: Minor
> Fix For: 3.8.0
>
>
> Throughout the process of a ZK to KRaft migration, the operator has the
> choice to revert back to ZK mode. Once this is done, there will be a copy of
> the metadata log on each broker in the cluster.
> In order to re-attempt the migration in the future, this metadata log needs
> to be deleted. This can be pretty burdensome to the operator for large
> clusters, especially since the log deletion must be done while the broker is
> offline.
> To improve this, we can automatically delete any metadata log present during
> startup of a ZK broker. In general, it is always safe to remove the metadata
> log from a KRaft or migrating ZK broker. The main impact is that this will
> delay the time it takes for the broker to be unfenced by the controller since
> it has to re-replicate the log. In the case of hybrid mode ZK brokers, there
> will be a delay in them receiving their first UpdateMetadataRequest from the
> controller (for the same reason -- delay in getting unfenced).
> The delayed startup should not affect the performance of the cluster, though
> it would increase the overall time required to do a rolling restart of the
> cluster.
> Once a broker restarts as KRaft, we will stop doing this automatic deletion.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
