wombatu-kun opened a new pull request, #18137: URL: https://github.com/apache/iceberg/pull/18137
Backport of #16791 to Flink 1.20, 2.1 and 2.2. The `ArrayDataWriter` and `MapDataWriter` in `FlinkParquetWriters` now build their element, key and value getters once at writer construction instead of once per row, and hand out a single reusable iterator per writer instead of allocating one per value. `ParquetValueWriters.RepeatedWriter.write` and `RepeatedKeyValueWriter.write` drain that iterator inside a single call and never retain it, a contract documented and tested by #18133, so the reuse is safe. `FlinkParquetWriters` in 2.1 and 2.2 was byte-identical to the 2.3 copy before #16791, so those two are direct copies of the merged file and this branch leaves all three identical again. The 1.20 copy differs from 2.3 only by the Variant imports, the `variant()` override and the `VariantWriter` class, none of which this change touches, so it takes the same four hunks at a constant offset, and after the port the 1.20-vs-2.3 delta is still exactly that Variant block. `TestFlinkParquetWriter` and `TestFlinkParquetReader` pass on all three versions; dropping the index reset from the reused iterators fails 9 of them on 1.20, covering arrays, maps, structs nested in both, and numeric and complex map keys. --- **AI Disclosure** - Model: Claude Opus 5 (1M context) - Platform/Tool: Claude Code - Human Oversight: fully reviewed - Prompt Summary: Backport the Flink Parquet array and map writer allocation change from #16791 to the Flink 1.20, 2.1 and 2.2 modules. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
