mxm opened a new pull request, #14044: URL: https://github.com/apache/iceberg/pull/14044
The tests https://github.com/apache/iceberg/blob/720ef99720a1c59e4670db983c951243dffc4f3e/flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/TestFlinkIcebergSinkExtended.java#L98 and https://github.com/apache/iceberg/blob/720ef99720a1c59e4670db983c951243dffc4f3e/flink/v2.0/flink/src/test/java/org/apache/iceberg/flink/sink/TestIcebergSink.java#L200 are often flaky, especially in the context of parallel test execution in #13675 where it almost always fails. The reason is that it uses a bounded source (`env.fromCollection(..)`) which does not wait for a checkpoint to get triggered before shutting down. If no checkpoint gets triggered before the job shuts down, no Iceberg snapshot gets created for the test table and it remains empty. The test then fails with the following exception: ``` TestIcebergSink > testTwoSinksInDisjointedDAG() > format=AVRO, parallelism=1, partitioned=true, isTableSchema=true FAILED java.lang.AssertionError: Expecting empty but was: [Record(1, right-aaa), Record(1, right-bbb), Record(1, right-ccc), Record(2, right-aaa), Record(2, right-bbb), Record(2, right-ccc), Record(3, right-aaa), Record(3, right-bbb), Record(3, right-ccc)] at org.apache.iceberg.flink.SimpleDataUtil.assertTableRecords(SimpleDataUtil.java:326) at org.apache.iceberg.flink.SimpleDataUtil.assertTableRows(SimpleDataUtil.java:264) at org.apache.iceberg.flink.sink.TestIcebergSink.testTwoSinksInDisjointedDAG(TestIcebergSink.java:295) ``` The exception is slightly misleading, which is why I'm changing the assert description alongside with the flaky test. The test has been tested also with #13675. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
