stevenzwu commented on PR #10567: URL: https://github.com/apache/iceberg/pull/10567#issuecomment-2318232558
> I found that I cannot directly use OneInputStreamOperatorTestHarness for testing. I noticed that we currently use it to test the StreamingReaderOperator, but the input for StreamingReaderOperator is FlinkInputSplit (corresponding to a CombinedScanTask), and it processes at least one completed split each time. This means it does not perform checkpoints while consuming partial splits. @zhongyujiang is this issue with the old `FlinkSource` or with the new FLIP-27 `IcebergSource`. `StreamingReaderOperator` is for the old `FlinkSource`, which we plan to deprecate in favor of the FLIP-27 source. See more context in the thread in dev ML. subject is "[DISCUSS] Flink 1.20: make FLIP-27 default in SQL and mark the old FlinkSource as deprecated" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
