pvary commented on code in PR #18602:
URL: https://github.com/apache/iceberg/pull/18602#discussion_r4347746023
##########
flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/DynamicRecordProcessor.java:
##########
@@ -160,14 +160,19 @@ public void collect(DynamicRecord inputData) {
isForward);
} else {
// Shuffled records with immediateUpdate=false go to the update side
output
+ // The schema, the spec and the row used for the key must describe the
same layout: the
+ // spec resolves partition source fields by ID in the schema, and the
schema reads them by
+ // position from the row. Use the table's layout only when both the
table schema and spec
+ // are known, otherwise stay with the record's own schema, spec and
row.
+ boolean keyByTable = foundSchema.resolvedTableSchema() != null &&
foundSpec != null;
int writerKey =
- hashKeyGenerator.generateKey(
- data,
- foundSchema.resolvedTableSchema() != null
- ? foundSchema.resolvedTableSchema()
- : data.schema(),
- foundSpec != null ? foundSpec : data.spec(),
- data.rowData());
+ keyByTable
Review Comment:
What if we just leave out the table schema and friends?
```
int writerKey = hashKeyGenerator.generateKey(data, null, null, null);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
