danielcweeks commented on PR #12721:
URL: https://github.com/apache/iceberg/pull/12721#issuecomment-2855780956
Hey @wypoon, I spent some time working on getting the test only approach
working, but ran into some other issues that I'm worried are even bigger
problems with our Hive compatibility story.
The issue is that while we are source compatible across hive2/3/4 (it
compiles), our artifacts and test artifacts are not compatible across these
versions. That means the original approach of having subprojects that all
compile and test against the single source tree, hides the binary
incompatibility. Since we are not producing separate artifacts, there's no
guarantee that the binaries produced will work in a hive3/4.
The main issue in Hive 3/4 is the `HiveSchemaUtil` which results in the
following if you use the binary artifacts across versions:
```java
java.lang.NoSuchMethodError: 'java.util.ArrayList
org.apache.hadoop.hive.serde2.typeinfo.StructTypeInfo.getAllStructFieldNames()'
at
org.apache.iceberg.hive.HiveSchemaConverter.convertType(HiveSchemaConverter.java:138)
at
org.apache.iceberg.hive.HiveSchemaConverter.convertInternal(HiveSchemaConverter.java:71)
at
org.apache.iceberg.hive.HiveSchemaConverter.convert(HiveSchemaConverter.java:55)
at
org.apache.iceberg.hive.HiveSchemaUtil.convert(HiveSchemaUtil.java:78)
at
org.apache.iceberg.hive.HiveSchemaUtil.convert(HiveSchemaUtil.java:56)
```
So either approach we use to test doesn't actually provide compatibility
other than knowing if you compiled for the specific version of Hive, the binary
artifacts would work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
