Thanks Mehul for driving it. +1 from my side. Best regards, Yuxia
----- 原始邮件 ----- 发件人: "Giannis Polyzos" <[email protected]> 收件人: "dev" <[email protected]> 发送时间: 星期日, 2025年 12 月 14日 下午 9:15:49 主题: Re: [Discuss] FIP-20: Introduce $changelog and $binlog Virtual Tables in Flink Engine Indeed this is an important feature, especially since auditing is becoming more and more crucial. Thanks for driving this. +1 from me Best, Giannis On Sun, 14 Dec 2025 at 1:33 PM, Jark Wu <[email protected]> wrote: > Thanks, Mehul! > > This is a very valuable feature for changelog auditing, and the FLIP > is already in great shape. > > +1 from my side. > > Best regards, > Jark > > On Mon, 8 Dec 2025 at 13:59, Mehul Batra <[email protected]> wrote: > > > > Hi all, > > > > Change Data Capture (CDC) is essential for real-time data processing, > > widely used in scenarios like incremental ETL, data synchronization, and > > audit logging. However, Fluss currently lacks a standardized mechanism to > > expose changelog and binlog data through Flink SQL, forcing users to > > implement custom change tracking solutions. > > > > Without native support, users face several challenges: > > > > - No SQL interface to query operation types (INSERT, UPDATE, DELETE) > > - No efficient access to change data for incremental processing > > - No consistent audit trails across applications > > - No visibility into before/after states for UPDATE operations > > > > To address this, I'd like to propose FIP-20: Introduce $changelog and > > $binlog Virtual Tables [1]. > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLUSS/FIP-20+Introduce+%24changelog+and+%24binlog+Virtual+Tables+in+Flink+Engine > > > > This proposal introduces: > > > > - *$changelog virtual table*: Flat schema with metadata columns ( > > _change_type, _log_offset, _commit_timestamp) exposing change > operations > > (+I, -U, +U, -D for PK tables; +A for Log tables) > > - *$binlog virtual table*: Nested schema with before/after ROW columns > > providing Debezium-style CDC format with event-level semantics (I, U, > D) > > - *Schema introspection*: Support for DESCRIBE and SHOW CREATE TABLE > on > > virtual tables > > - *Broad table support*: $changelog works for both Primary Key tables > > and Log tables; $binlog for Primary Key tables only > > > > This follows the existing $lake virtual table pattern, ensuring > consistency > > across Fluss's virtual table design. > > > > Any feedback and suggestions are welcome! > > > > Best regards, > > Mehul >
