manuzhang closed pull request #10996: Remove Hive 2
URL: https://github.com/apache/iceberg/pull/10996
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: is
github-actions[bot] commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2585490637
This pull request has been marked as stale due to 30 days of inactivity. It
will be closed in 1 week if no further activity occurs. If you think that’s
incorrect or this pul
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2537550479
We have a [consensus on the dev
list](https://lists.apache.org/thread/jfcqfw9vhq4j7h0kwnlf338jgyzcq8s4) to drop
hive-runtime and upgrade to Hive 4. I've submitted
https://github.com/
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2485864509
I have learned that there are no Hive 3 releases are planned. So these tests
will not work with Hive3.
I would like to see the Hive connector upgraded to Hive 4 which supports
Java
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2487438171
@pvary @nastra @Fokko @gaborkaszab I've sent out a discussion email. Please
share your thoughts there.
--
This is an automated message from the Apache Git Service.
To respond to th
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2485757937
Let me consult with the Hive folks and come back soon with a proposal.
Thanks,
Peter
--
This is an automated message from the Apache Git Service.
To respond to the message, please
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2482780914
> The failed tests list [..]
> Do we have these testing for Hive 3 on main branch now?
I think these tests should be running fine with Java 8. The issue is that
with Java 11 we c
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2482615446
@pvary The failed tests list
- TestHiveIcebergStorageHandlerLocalScan
- TestHiveIcebergStorageHandlerNoScan
- TestHiveIcebergStorageHandlerTimezone
- TestHiveIcebergSto
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2482012167
> @pvary @nastra @Fokko Given Hive 3.1 is already broken on JDK 11+ before
this PR, how about skipping these failed tests not to block removing Hive 2? I
think upgrading to Hive 4 is a bi
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2481719211
@pvary @nastra @Fokko
Given Hive 3.1 is already broken on JDK 11+ before this PR, how about
skipping these failed tests not to block removing Hive 2? I think upgrading to
Hive 4 is
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2480424096
> It's been back-ported to hive 3.1, but there's no new release yet. @pvary
@nastra thoughts?
Basically the missing release means that there is no Hive 3 with Java 11
(contrary to
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2479686648
> > I expect that there will be users trying to use HiveCatalog with old
Hive versions.
>
> @pvary We don't expect them to use iceberg 1.8+ with old Hive versions,
right?
I
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2479763958
> It's been back-ported to hive 3.1, but there's no new release yet. @pvary
@nastra thoughts?
What about using Hive4 instead?
--
This is an automated message from the Apache Git
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2479011158
6 tests in `iceberg-mr` failed due to
[HIVE-21584](https://issues.apache.org/jira/browse/HIVE-21584).
```
TestHiveIcebergWithHiveAutogatherEnable > initializationError FAILED
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2478998374
> I expect that there will be users trying to use HiveCatalog with old Hive
versions.
@pvary We don't expect them to use iceberg 1.8+ with old Hive versions,
right?
--
This
manuzhang commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1843854663
##
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##
@@ -59,12 +57,12 @@ public class HiveIcebergInputFormat extends
MapredIcebergInputFo
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1843834812
##
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##
@@ -59,12 +57,12 @@ public class HiveIcebergInputFormat extends
MapredIcebergInputForma
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1843836878
##
mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java:
##
@@ -213,11 +212,11 @@ private static final class IcebergRecordReader extends
RecordRea
pvary commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2478468746
I expect that there will be users trying to use HiveCatalog with old Hive
versions.
Do we want to (could we) throw a meaningful exception when this happens?
Maybe we can throw an
pvary commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1843517724
##
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerNoScan.java:
##
@@ -770,9 +769,7 @@ public void testIcebergAndHmsTableProperties() throws
E
pvary commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1843512697
##
mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java:
##
@@ -479,6 +446,17 @@ private CloseableIterable newOrcIterable(
return applyResidu
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1843196936
##
.github/workflows/hive-ci.yml:
##
@@ -66,34 +66,34 @@ concurrency:
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
jobs:
- hive2-tests:
+ mr-
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1842669520
##
mr/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##
@@ -63,19 +61,15 @@ public class HiveIcebergInputFormat extends
MapredIcebergInputForma
nastra commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2472951142
> Remove Hive dependency from iceberg-mr and merge all Hive related codes
into iceberg-hive3
@manuzhang can we maybe do this in a separate step? I think the scope of
this PR shoul
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1839800670
##
hive3/src/main/java/org/apache/iceberg/mr/hive/HiveIcebergInputFormat.java:
##
@@ -63,19 +61,15 @@ public class HiveIcebergInputFormat extends
MapredIcebergInputFo
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1839813669
##
hive3/src/main/java/org/apache/iceberg/mr/hive/Deserializer.java:
##
Review Comment:
why are all of these files being moved from `iceberg-mr`?
--
This is a
manuzhang commented on PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#issuecomment-2466177889
@nastra This is ready for review now after 1.7.0 release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
manuzhang commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1753036975
##
mr/src/test/java/org/apache/iceberg/mr/hive/serde/objectinspector/TestIcebergObjectInspector.java:
##
@@ -99,14 +98,9 @@ public void testIcebergObjectInspector()
manuzhang commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1753036975
##
mr/src/test/java/org/apache/iceberg/mr/hive/serde/objectinspector/TestIcebergObjectInspector.java:
##
@@ -99,14 +98,9 @@ public void testIcebergObjectInspector()
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1750200212
##
mr/src/main/java/org/apache/iceberg/mr/hive/serde/objectinspector/IcebergObjectInspector.java:
##
@@ -27,33 +27,23 @@
import org.apache.hadoop.hive.serde2.typeinfo
nastra commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1750198331
##
mr/src/main/java/org/apache/iceberg/mr/hive/serde/objectinspector/IcebergObjectInspector.java:
##
@@ -27,33 +27,23 @@
import org.apache.hadoop.hive.serde2.typeinfo
manuzhang commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r1734916650
##
mr/build.gradle:
##
@@ -52,8 +52,8 @@ project(':iceberg-mr') {
exclude group: 'org.pentaho' // missing dependency
exclude group: 'org.slf4j', modul
BsoBird commented on code in PR #10996:
URL: https://github.com/apache/iceberg/pull/10996#discussion_r173258
##
mr/build.gradle:
##
@@ -52,8 +52,8 @@ project(':iceberg-mr') {
exclude group: 'org.pentaho' // missing dependency
exclude group: 'org.slf4j', module:
33 matches
Mail list logo
