showuon commented on PR #16653: URL: https://github.com/apache/kafka/pull/16653#issuecomment-2249560355
> While reviewing this PR, I noticed that the current way of applying the topic config is kind of confusing and lead to lot of `if-else` cases. If we apply the config in below manner, then most of the issues will go away: > > 0. The default value of `remote.log.disable.policy` should be null. And, it should take only one value: "retain" > > 1. To gracefully disable the remote storage for a topic, the user should set the `remote.log.disable.policy` to `retain`, then we stop the copy and follower tasks. The expiration task will be running in the background. The consumer will be able to read the archived data from remote storage and the remote data gets deleted asynchronously. In this state, `remote.storage.enable` should still be set to `true` as the archived data is still served from remote storage. > > 2. Once all the remote data gets deleted async (ie after the retention time/size gets passed), then the user can set the `remote.storage.enable` to `false` to stop the respective RLM and RLMM resources for those topics. > > 3. If the user directly sets the `remote.storage.enable` from `true` to `false`, then we should ungracefully delete all the remote log segments and stop the respective resources similar to the `remote.log.disable.policy` as `delete`. > > 4. If the user wants to resume the copy tasks after "retain" disable policy, then `remote.log.disable.policy` can be reset back to `null`. > > > If the above approach looks good, we can append them to the KIP-950 and ask in the voting thread for any objections. I think re-thinking the original design is a good call. Let me think about your proposal clearly and get back to you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
