swapna267 commented on PR #12116:
URL: https://github.com/apache/iceberg/pull/12116#issuecomment-2620523105

   **Background:**
   1. Creation of dynamic Iceberg table in Flink Catalog using the underlying 
physical Iceberg table using LIKE clause.
   
   Currently (without the changes in PR),  create table in flink catalog works 
by configuring flink connector as described in,
   
[flink-connector](https://iceberg.apache.org/docs/nightly/flink-connector/#flink-connector
 )
   
   But that needs user to provide the schema for the table. A way to tackle 
that is to do create table LIKE using below DDL.
   
   ```
   CREATE TABLE table_wm (
         eventTS AS CAST(t1 AS TIMESTAMP(3)),
         WATERMARK FOR eventTS AS SOURCE_WATERMARK()
   ) WITH (
     'connector'='iceberg',
     'catalog-name'='iceberg_catalog',
     'catalog-database'='testdb',
     'catalog-table'='t'
   ) LIKE iceberg_catalog.testdb.t;
   ```
   
   Options like connector, catalog-name, catalog-database, catalog-table need 
to be duplicated as Iceberg FlinkCatalog doesn't return any catalog related 
properties during getTable. This PR addresses the issue by including these 
properties when getTable is called , which will be used by Flink when creating 
table in Flink Catalog.
   
   
   2.  Iceberg Source to support Source Watermark
   As raised in https://github.com/apache/iceberg/issues/10219,   Source 
watermark implemented as part of 
https://iceberg.apache.org/docs/nightly/flink-queries/#emitting-watermarks 
cannot be used in Flink window functions. 
   
   
[SupportSourceWatermark.java](https://github.com/apache/flink/blob/release-1.20/flink-table/flink-table-common/src/main/java/org/apache/flink/table/connector/source/abilities/SupportsSourceWatermark.java)
 interface to push down watermark to source. 
   
   Here we are falling back to read options implemented in 
https://github.com/apache/iceberg/pull/9346 , to configure the watermark column 
on Iceberg Source. 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to