stevenzwu commented on code in PR #10748:
URL: https://github.com/apache/iceberg/pull/10748#discussion_r1693308647
##########
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestFlinkSourceConfig.java:
##########
@@ -49,11 +48,11 @@ public void testFlinkHintConfig() {
@TestTemplate
public void testReadOptionHierarchy() {
- // TODO: FLIP-27 source doesn't implement limit pushdown yet
- assumeThat(useFlip27Source).isFalse();
-
getTableEnv().getConfig().set(FlinkReadOptions.LIMIT_OPTION, 1L);
List<Row> result = sql("SELECT * FROM %s", TABLE);
+ // Note that this query doesn't have the limit clause in the SQL.
+ // This assertions works because limit is pushed down to the reader and
+ // reader parallelism is 1.
Review Comment:
That is exactly what I am trying to clarify here, because the SQL query has
no limit clause. if the source parallelism is 4, there could be 4 readers and
each may emit 1 record. Note that limit pushdown is not guarantee that source
only emit the limited number of record. Source only needs to try its best to
break/stop early. The SQL limit clause and the SQL engine does the final result
limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
