sannaroby commented on issue #10147:
URL: https://github.com/apache/iceberg/issues/10147#issuecomment-2058495395
Hi @pvary, thanks for your reply.
We're using the HASH distribution mode and this is an extract from our flink
job:
```
SingleOutputStreamOperator<Row> mainFunction = env.addSource(kinesisSource)
.keyBy(inputMessage-> inputMessage.getKey())
.window(TumblingProcessingTimeWindows.of(Time.seconds(10)))
.process(new InputMessageToRowProcessFunction(), rowTypeInfo);
FlinkSink.forRow(mainFunction, loadedTableSchema)
.tableLoader(tableLoader)
.table(icebergTable)
.equalityFieldColumns(List.of("messageKey", "messageTimestamp"))
.distributionMode(DistributionMode.HASH)
.append();
```
The autoscaler didn't change the upstream operator parallelism (the
"InputMessageToRowProcessFunction"), because it judged it to be unnecessary.
What do you mean for "rebalance step" ?
Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
