Hi Yunhong, Thank you for driving this very useful initiative—it will significantly improve the stability of data consumption and the overall user experience. I generally agree with most aspects of the design, but I have a few minor questions:
1. When Flink enumerates the snapshots it needs to subscribe to and attempts to subscribe, what is the expected behavior if a snapshot gets deleted just before the subscription request is sent? (This scenario might occur in certain edge cases.) 2. Retaining only one snapshot by default could be risky. Currently, neither the CoordinatorServer nor the TabletServer can fully guarantee that snapshots registered in ZooKeeper are always reliable and valid (although this is the design intention, there might still be undiscovered bugs). 3. I didn’t see any API for updating subscriptions. If a Flink job experiences a prolonged failover for some reason, would the unread snapshots expire as a result? 4. Related to the previous question: Is the TTL of a snapshot calculated from the time it was registered, or from the last time a consumer updated its subscription? Best regards, Yang yunhong Zheng <[email protected]> 于2025年12月11日周四 22:40写道： > Hi all, > > Currently, for Fluss PrimaryKey Table, the number of kv snapshots retained > per bucket is controlled by the server > option `kv.snapshot.num-retained` (default value: 1). If this value is set > too small, Kv snapshots that are being actively consumed may be deleted > while a consumer is still consuming them. > > This case will cause a Flink job which read PrimaryKey table fail and > cannot be restarted from its previous state. > > To avoid this case, the fluss server needs to be aware of which consumers > are actively consuming the corresponding kv snapshots, and can not delete > these kv snapshots that are currently being consumed. > > So, I'd like to propose FIP-22: Support Kv Snapshot Consumer[1]. > > Any feedback are suggestions on this proposal are welcome! > > [1]: > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-22+Support+Kv+Snapshot+Consumer > > Regards, > Yunhong >
