j1wonpark opened a new issue, #4377: URL: https://github.com/apache/amoro/issues/4377
### Search before asking - [x] I have searched in the issues and found no similar issues. ### What would you like to be improved? AMS, the optimizers and the mixed-format connectors all share the single root `iceberg.version` (1.8.1). Every Iceberg bump therefore has to be paid twice: once for the maintenance path (planner, rewrite executors, snapshot expiration, orphan clean) and once for every mixed-format engine connector. The connector side is where the cost actually lands, because each Iceberg release retires engine bindings (1.9 drops Spark 3.3, 1.10 drops Flink 1.18, 1.11 requires JDK 17), so upgrading Iceberg for maintenance forces the same engine drops on users of the mixed-format connectors. The maintenance service and the mixed-format connectors run in different processes (AMS / optimizer JVMs vs. Spark, Flink and Trino jobs), so they do not need the same Iceberg version. ### How should we improve? Let the maintenance path (AMS, optimizers, dist) select its own Iceberg version while `amoro-format-iceberg`, `amoro-mixed-hive` and the engine connectors stay on the current baseline. The attached PR keeps every module in place and only - adds a `maintenance-iceberg.version` property that AMS, optimizer and dist use for their Iceberg (and Parquet) dependency management, - switches the two Parquet reader calls removed in Iceberg 1.9 to the equivalent form that exists in both 1.8.1 and 1.10.x, - runs the shared mixed-format I/O tests once more from the AMS module, so they are executed against the Iceberg version AMS actually ships. One decision is needed alongside the PR: the newer Parquet required by Iceberg 1.10 does not run on Hadoop 2, so AMS would require Hadoop 3 while the connectors keep their Hadoop 2 build. Where new V3-only maintenance code should live, given that the shared modules stay on the lower API, is a follow-up. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's Code of Conduct -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
