stevenzwu commented on code in PR #10748:
URL: https://github.com/apache/iceberg/pull/10748#discussion_r1693308647
##########
flink/v1.19/flink/src/test/java/org/apache/iceberg/flink/source/TestFlinkSourceConfig.java:
##########
@@ -49,11 +48,11 @@ public void testFlinkHintConfig() {
@TestTemplate
public void testReadOptionHierarchy() {
- // TODO: FLIP-27 source doesn't implement limit pushdown yet
- assumeThat(useFlip27Source).isFalse();
-
getTableEnv().getConfig().set(FlinkReadOptions.LIMIT_OPTION, 1L);
List<Row> result = sql("SELECT * FROM %s", TABLE);
+ // Note that this query doesn't have the limit clause in the SQL.
+ // This assertions works because limit is pushed down to the reader and
+ // reader parallelism is 1.
Review Comment:
That is correct. That is exactly what I am trying to clarify here, because
the SQL query has no limit clause. if the source parallelism is 4, there could
be 4 readers and each may emit 1 record. Note that limit pushdown is not
guarantee that source only emit the limited number of record. Source only needs
to try its best to break/stop early. The SQL limit clause and the SQL engine
does the final result limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
