guixiaowen opened a new issue, #12753:
URL: https://github.com/apache/iceberg/issues/12753
### Feature Request / Improvement
I am working on converting a large number of Hive tables to Iceberg tables.
I’ve found that some syntax supported by Hive tables is transformed into a
different form in the Iceberg context. For example, the Hive table syntax for
deleting partitions is:
hive sql：
`
ALTER TABLE %s DROP PARTITION(pt='123', hour='02'), PARTITION(pt='123',
hour='03');
`
iceberg sql
`
DELETE FROM %s WHERE pt = '123' AND hour = '01';
`
To accelerate the migration and ensure compatibility with existing user SQL,
we aim to keep Hive syntax working even after switching to Iceberg tables. This
is achieved by transforming Hive-style syntax into Iceberg syntax during the
SQL parsing phase, allowing historical Hive job scenarios to continue
functioning correctly.
EG：
before this pr
`
explain extended ALTER TABLE %s DROP PARTITION(pt='123', hour='02'),
PARTITION(pt='123', hour='03');
== Parsed Logical Plan ==
'DropPartitions [unresolvedpartitionspec((pt,123), (hour,01), None)], false,
false
+- 'UnresolvedTable [xxx, ALTER TABLE ... DROP PARTITION ..., Please use
ALTER VIEW instead.
== Analyzed Logical Plan ==
AlterTableDropPartitionCommand `xxx`, [Map(pt -> 123, hour -> 01)], false,
false, false
== Optimized Logical Plan ==
AlterTableDropPartitionCommand `xxx`, [Map(pt -> 123, hour -> 01)], false,
false, false
== Physical Plan ==
Execute AlterTableDropPartitionCommand
+- AlterTableDropPartitionCommand `xxx`, [Map(pt -> 123, hour -> 01)],
false, false, false
`
before this pr
explain extended ALTER TABLE %s DROP PARTITION(pt='123', hour='02'),
PARTITION(pt='123', hour='03');
`
== Parsed Logical Plan ==
'DeleteFromIcebergTable (('pt = 123) AND (('hour = 01) OR ('hour = 02)))
+- 'UnresolvedRelation [xxx], [], false
== Analyzed Logical Plan ==
DeleteFromIcebergTable ((pt#377 = 123) AND ((hour#378 = 01) OR (hour#378 =
02)))
:- RelationV2[a#375, b#376, pt#377, hour#378]
ICEBERG.bigdata_qa.test_drop_partition_iceberg_singal
+- ReplaceIcebergData RelationV2[a#375, b#376, pt#377, hour#378] ICEBERG.xxx
+- Filter NOT (((pt#377 = 123) AND ((hour#378 = 01) OR (hour#378 = 02)))
<=> true)
+- RelationV2[a#375, b#376, pt#377, hour#378, _file#381, _pos#382L]
ICEBERG.xxx
== Optimized Logical Plan ==
DeleteFromIcebergTable ((pt#377 = 123) AND ((hour#378 = 01) OR (hour#378 =
02)))
+- RelationV2[a#375, b#376, pt#377, hour#378] ICEBERG.xxx
== Physical Plan ==
DeleteFromTable ICEBERG.xxx, [EqualTo(pt,123),
Or(EqualTo(hour,01),EqualTo(hour,02))],
org.apache.spark.sql.execution.datasources.v2.ExtendedDataSourceV2Strategy$$Lambda$4138/777836889@73f506d7
`
### Query engine
Spark
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with
guidance from the Iceberg community
- [x] I cannot contribute this improvement/feature at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
