yusufyildiz34 opened a new pull request, #18508: URL: https://github.com/apache/iceberg/pull/18508
`FlinkSourceFilter` re-evaluates pushed-down filters per row through `RowDataWrapper`, which had no getters for ARRAY and MAP. It returned Flink's `ArrayData`/`MapData` where Iceberg expects `List`/`Map`, so `IS NULL`/`IS NOT NULL` on these columns failed with a `ClassCastException`. This adds ARRAY and MAP getters that convert elements to Iceberg's internal representation, mirroring `StructRowData`. Collections are copied rather than wrapped because readers reuse arrays across rows and `StructLikeUtil.copy` keeps collection values by reference. Converters are built once per wrapper and run only when such a column is read. VARIANT has the same gap and will be a follow-up, as will backports to Flink 1.20 and 2.2. Closes #18504 Test plan: `TestFlinkSourceFilter` covers `IS NULL`/`IS NOT NULL` on list, map and nested-list columns; new `TestRowDataWrapper` cases cover element conversion for strings, timestamps, structs, map decimals and unknown-typed elements. All nine fail on main and pass with this change. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: fully reviewed - Prompt Summary: Investigate #18504, choose a design from project history (`StructRowData`, #16791), implement ARRAY/MAP support in `RowDataWrapper` for Flink 2.3, and add regression tests. The code and tests are generated; the design decisions were reviewed by the author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
