yhyyz opened a new pull request, #18610: URL: https://github.com/apache/iceberg/pull/18610
`EqualityConvertReader` reads data files and equality delete files as generic `Record`s, where temporal values are `java.time` objects (`LocalDate`, `LocalTime`, `LocalDateTime`, `OffsetDateTime`). It then passes these records straight to `StructLikeSerializer.serializeKey`, which calls `Conversions.toByteBuffer` and expects Iceberg's internal representation (`int` days for `DATE`, `long` micros for `TIME`/`TIMESTAMP`/`TIMESTAMPTZ`). As a result, `ConvertEqualityDeletes` fails on every cycle with a `ClassCastException` when the equality fields include a temporal column. Because the partition source columns must be part of the equality fields, this affects every table partitioned by a date column. This change wraps the records with `InternalRecordWrapper` before serializing the key, in both the data file path and the equality delete file path. ### Testing Added `TestEqualityConvertReader#serializesTemporalEqualityFieldsFromDataAndDeleteFiles`. It uses a v3 table partitioned by a `DATE` column with `DATE`, `TIME`, `TIMESTAMP` and `TIMESTAMPTZ` equality fields, and checks that the key serialized from the data file equals the key serialized from the equality delete file. Without the fix, the test fails with the `ClassCastException`. This PR changes Flink 2.3 only. I will open the backport to 1.20 and 2.2 as a separate PR. Fixes #18609 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
