pan3793 commented on issue #12878: URL: https://github.com/apache/iceberg/issues/12878#issuecomment-3086459275
@kevinjqliu The Spark's multi HMS versions support is only applicable for Spark's built-in Hive catalog, Iceberg's Hive catalog can NOT benefit from it. I have repeatedly explained to different people recently, if it is confusing, maybe we should clarify it in the docs. https://github.com/apache/iceberg/issues/12878#issuecomment-2834070948 https://github.com/apache/iceberg/pull/12721#issuecomment-2995740677 https://github.com/apache/iceberg/pull/12721#issuecomment-2995850122 @wypoon I appreciate your efforts in making it compatible with more HMS versions, but the approach does not solve OSS Spark users' requirement - use OSS Spark 3.5/4.0 with built Hive 2.3 jars to communicate with HMS 4.0, it requires that the user have a custom Spark that upgrades their built-in Hive to 4.0 first, this requires huge work. TBH, Iceberg Hive catalog only uses a few Thrift API to talk with HMS, if the Hive community does not want to keep those Thrift API used by the old Hive client, maybe building a thin HMS client from scratch (starting from generating code from Thrift IDL) is an eligible way, this is what Trino does. https://github.com/trinodb/trino/issues/15921 https://github.com/trinodb/hive-thrift -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
