Re: [I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-25 Thread via GitHub
pvary commented on issue #11573: URL: https://github.com/apache/iceberg/issues/11573#issuecomment-2499768751 Could you partition your table on `bucket(id, ...)`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-21 Thread via GitHub
pvary commented on issue #11573: URL: https://github.com/apache/iceberg/issues/11573#issuecomment-2491203879 > But my use case requires upsert, so in this scenario, using status as the partition key is not suitable, right? The problem is not with the partitioning. The problem is that

Re: [I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-20 Thread via GitHub
a8356555 commented on issue #11573: URL: https://github.com/apache/iceberg/issues/11573#issuecomment-2489959870 > What are the records generated by the MySQL CDC connector? > > You are using upsert mode in FlinkSink. > > In upsert mode when an update happens, Flink expects an un

Re: [I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-20 Thread via GitHub
a8356555 commented on issue #11573: URL: https://github.com/apache/iceberg/issues/11573#issuecomment-2489959318 But my use case requires upsert, so in this scenario, using status as the partition key is not suitable, right? -- This is an automated message from the Apache Git Service. To r

Re: [I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-18 Thread via GitHub
pvary commented on issue #11573: URL: https://github.com/apache/iceberg/issues/11573#issuecomment-2482029220 What are the records generated by the MySQL CDC connector? You are using upsert mode in FlinkSink. In upsert mode when an update happens, Flink expects an unchanged prima

[I] Handling Updates on Partition Columns in Iceberg with Flink CDC [iceberg]

2024-11-17 Thread via GitHub
a8356555 opened a new issue, #11573: URL: https://github.com/apache/iceberg/issues/11573 ### Apache Iceberg version 1.5.2 ### Query engine Athena ### Please describe the bug 🐞 Hi, I'm using MySQL Flink CDC with Iceberg 1.5.2 and Flink 1.16. I have a t