jini-lee opened a new issue, #7947:
URL: https://github.com/apache/iceberg/issues/7947
### Apache Iceberg version
1.3.0 (latest release)
### Query engine
Hive
### Please describe the bug 🐞
Can't read table's data on hive created by spark-sql but data exists.
How can i fix it ?
Below is the work history I tried.
Thanks.
**Version Info**
```
Spark - 3.1.2
Hive - 3.1
```
**Step**
1) I started spark-sql shell.
```
spark-sql \
--packages org.apache.iceberg:iceberg-spark-runtime-3.1_2.12:1.3.0 \
--conf spark.sql.catalog.test=org.apache.iceberg.spark.SparkCatalog \
--conf spark.sql.catalog.test.type=hadoop \
--conf spark.sql.catalog.test.warehouse=hdfs://my-hdfs/warehouse
spark-sql> add jar
hdfs://my-hdfs/user/test/tmp/iceberg-hive-runtime-1.3.0.jar;
```
2) Then, create table and insert some data like below. I did check the table
be created on warehouse path.
```
spark-sql> CREATE TABLE test.db.test (id bigint, data string) USING iceberg
TBLPROPERTIES ('iceberg.engine.hive.enabled' = 'true' );
Time taken: 0.319 seconds
spark-sql> INSERT INTO test.db.test VALUES (1, 'a'), (2, 'b'), (3, 'c');
Time taken: 7.096 seconds
spark-sql> select * from test.db.test;
1 a
2 b
3 c
Time taken: 1.766 seconds, Fetched 3 row(s)
```
3) For query on hive, create external table on beeline.
```
0: jdbc:hive2://hiveserver > add jar
hdfs://my-hdfs/user/test/tmp/iceberg-hive-runtime-1.3.0.jar;
0: jdbc:hive2://hiveserver > CREATE DATABASE test;
0: jdbc:hive2://hiveserver > use test;
0: jdbc:hive2://hiveserver > CREATE EXTERBAL TABLE test (id bigint, data
string)
. . . .> STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
. . . .> LOCATION 'hdfs://my-hdfs/warehouse/db/test';
0: jdbc:hive2://hiveserver > DESCRIBE FORMATTED test;
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| col_name | data_type
| comment |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
| id | bigint
| from deserializer |
| data | string
| from deserializer |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | test
| NULL |
| OwnerType: | USER
| NULL |
| Owner: | linead
| NULL |
| CreateTime: | Fri Jun 30 18:08:06 JST 2023
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Location: | hdfs://my-hdfs/warehouse/db/test
| NULL |
| Table Type: | EXTERNAL_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | EXTERNAL
| TRUE |
| | bucketing_version
| 2 |
| | current-schema
|
{\"type\":\"struct\",\"schema-id\":0,\"fields\":[{\"id\":1,\"name\":\"id\",\"required\":false,\"type\":\"long\"},{\"id\":2,\"name\":\"data\",\"required\":false,\"type\":\"string\"}]}
|
| | engine.hive.enabled
| true |
| | external.table.purge
| TRUE |
| | metadata_location
|
hdfs://my-hdfs/warehouse/db/test/metadata/00000-f4e2e9d0-b7ad-4f25-b333-999e1acf0f00.metadata.json
|
| | snapshot-count
| 0 |
| | storage_handler
| org.apache.iceberg.mr.hive.HiveIcebergStorageHandler |
| | table_type
| ICEBERG |
| | transient_lastDdlTime
| 1688116086 |
| | uuid
| b5812155-41a1-4200-9288-953639ec4771 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.iceberg.mr.hive.HiveIcebergSerDe | NULL
|
| InputFormat: |
org.apache.iceberg.mr.hive.HiveIcebergInputFormat | NULL
|
| OutputFormat: |
org.apache.iceberg.mr.hive.HiveIcebergOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | 0
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
+-------------------------------+----------------------------------------------------+----------------------------------------------------+
3) Result of select query on beeline
```
0: jdbc:hive2://hiveserver > SELECT * FROM test;
+----------+------------+
| test.id | test.data |
+----------+------------+
+----------+------------+
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
