swapna267 commented on code in PR #15279:
URL: https://github.com/apache/iceberg/pull/15279#discussion_r2795466131
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/IcebergTableSink.java:
##########
@@ -184,4 +226,55 @@ public String asSummaryString() {
public void applyOverwrite(boolean newOverwrite) {
this.overwrite = newOverwrite;
}
+
+ private DataStreamSink<?> createDynamicIcebergSink(DataStream<RowData>
dataStream) {
+ TableCreator tableCreator = createTableCreator();
+ DynamicRecordGenerator<RowData> generator =
+ createDynamicRecordGenerator(dynamicRecordGeneratorImpl);
+
+ DynamicIcebergSink.Builder<RowData> builder =
+ DynamicIcebergSink.forInput(dataStream)
+ .generator(generator)
+ .catalogLoader(catalogLoader)
+ .setAll(writeProps)
+ .tableCreator(tableCreator)
+ .flinkConf(readableConfig);
+
+ return builder.append();
+ }
+
+ private TableCreator createTableCreator() {
+ final Map<String, String> tableProperties =
+ org.apache.iceberg.util.PropertyUtil.propertiesWithPrefix(writeProps,
"table.props.");
Review Comment:
Yes right. When doing CREATE TABLE in flink catalog , we pass in catalog
configuration here.
`table.props` prefix is used to separate out the physical Iceberg table
properties.
Basic documentation about the connector is here,
https://iceberg.apache.org/docs/nightly/flink-connector/
Once we have all functionality (dynamic record generator impl is coming in
next PR), will add details there.
I combined this in existing test case ,
https://github.com/swapna267/iceberg/blob/bd2d500f07fb24d05111b6dabc9a8e77637a922c/flink/v2.1/flink/src/test/java/org/apache/iceberg/flink/TestIcebergConnector.java#L393
I can pull it out into another one if we think it's required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
