wypoon commented on PR #12721: URL: https://github.com/apache/iceberg/pull/12721#issuecomment-2963811500
@danielcweeks thank you for your response. In this PR, I have not changed the runtime jars that bundle the iceberg-hive-metastore classes. They still bundle the classes that are built against Hive 2.3. In the last community sync, it seemed to me that there was agreement to 1. produce iceberg-hive-metastore jars for Hive 2, 3, and 4; 2. not produce multiple versions of engine runtime jars for different Hive versions; 3. eventually remove the iceberg-hive-metastore classes from engine runtime jars. As 3 is a breaking change, @rdblue said that we should wait till Iceberg 2.0 to do it, but that in the meantime, it is ok to produce engine runtime jars that bundle iceberg-hive-metastore classes built against Hive 2 (status quo), as that doesn't break anybody. I do appreciate your point that an iceberg-hive-metastore jar for Hive 4 (in this PR, that would be iceberg-hive4-metastore.jar) cannot be consumed with the engine runtime jars until 3 happens. However, there is value in this work going in now, as there is a burden to maintain an open PR while changes to the iceberg-hive-metastore occur (which does happen). Also, folks who do want to use Hive 4 can modify the engine build files to build runtime jars themselves by switching the dependency. 1 and 3 can really be done in separate changes, as 3 may in fact be part of a bigger change where not only iceberg-hive-metastore classes but classes for other catalogs get uncoupled/unbundled from the engine runtime jars. I think testing engines against different versions of Hive should be resolved separately. The community can decide for each engine which version/s of Hive they would like to test. I put up a PoC here just for testing some engines against Hive 4: https://github.com/apache/iceberg/pull/13262. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
