wypoon commented on PR #12681: URL: https://github.com/apache/iceberg/pull/12681#issuecomment-2777659880
@pvary I replied on the dev list. > I really would like to see HiveCatalog testing against Hive4, but Spark has constraints around testing newer Hive versions which we need to solve, or wait until all of the supported Spark version will have an updated Hive. > I think testing Flink and Spark with a Hive 3/Hive 4 metastore should be a separate PR. This is the first step. I also think this step makes it possible for the Hive project to drop its fork of this module from the Hive repo. I also do not think we should wait until all supported Spark versions support Hive 4, for this is never going to happen for Spark 3.4 or even Spark 3.5 (which is a LTS version and which I expect Iceberg to support for a while). > OTOH, if possible, I would like to see only a single TestHiveMetastore class. Could we do some changes internally to the class, which based on the classes available on the classpath, or some configuration would use the correct Hive versions (maybe with DynMethods)? The problem is with the HMSHandler class. Yes, I can use `DynConstructors` to construct the `HMSHandler` using the classname based on the Hive version, but `TestHiveMetastore` holds an instance of `HMSHandler` in a field, `baseHandler`, and in `TestHiveMetastore::stop()`, it calls `baseHandler.shutdown()`. We need to declare this field, but the type of the field is `org.apache.hadoop.hive.metastore.HMSHandler` in Hive 4 and `org.apache.hadoop.hive.metastore.HiveMetaStore.HMSHandler` in Hive 2 and 3. I don't see how to declare that field in a way independent of the Hive version. The interface that `HMSHandler` implements, `IHMSHandler`, does not have a `shutdown()` method. I need to call that method on `HMSHandler`, not `IHMSHandler`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
