Xuanwo opened a new issue, #790:
URL: https://github.com/apache/iceberg-rust/issues/790
The table generated in this way:
```sql
CREATE OR REPLACE ICEBERG TABLE lineitem (
l_orderkey BIGINT,
l_partkey BIGINT,
l_suppkey BIGINT,
l_linenumber INT,
l_quantity DECIMAL(15,2),
l_extendedprice DECIMAL(15,2),
l_discount DECIMAL(15,2),
l_tax DECIMAL(15,2),
l_returnflag STRING,
l_linestatus STRING,
l_shipdate DATE,
l_commitdate DATE,
l_receiptdate DATE,
l_shipinstruct STRING,
l_shipmode STRING,
l_comment STRING
)
CATALOG = 'SNOWFLAKE'
EXTERNAL_VOLUME = 'iceberg_external_volume'
BASE_LOCATION = 'lineitem'
CATALOG_SYNC = 'icebergPolaris';
INSERT INTO lineitem
SELECT * FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF100.LINEITEM;
```
I'm using iceberg-rust to read data from given table via polaris catalog:
```sql
select * from lineitem limit 1;
```
Got the following error:
```
1001=>iceberg data stream read: DataInvalid => Parquet schema table {
1: L_ORDERKEY: optional long
2: L_PARTKEY: optional long
3: L_SUPPKEY: optional long
4: L_LINENUMBER: optional int
5: L_QUANTITY: optional decimal(4,2)
6: L_EXTENDEDPRICE: optional decimal(8,2)
7: L_DISCOUNT: optional decimal(3,2)
8: L_TAX: optional decimal(3,2)
9: L_RETURNFLAG: optional string
10: L_LINESTATUS: optional string
11: L_SHIPDATE: optional date
12: L_COMMITDATE: optional date
13: L_RECEIPTDATE: optional date
14: L_SHIPINSTRUCT: optional string
15: L_SHIPMODE: optional string
16: L_COMMENT: optional string
}
and Iceberg schema table {
1: L_ORDERKEY: optional long
2: L_PARTKEY: optional long
3: L_SUPPKEY: optional long
4: L_LINENUMBER: optional int
5: L_QUANTITY: optional decimal(15,2)
6: L_EXTENDEDPRICE: optional decimal(15,2)
7: L_DISCOUNT: optional decimal(15,2)
8: L_TAX: optional decimal(15,2)
9: L_RETURNFLAG: optional string
10: L_LINESTATUS: optional string
11: L_SHIPDATE: optional date
12: L_COMMITDATE: optional date
13: L_RECEIPTDATE: optional date
14: L_SHIPINSTRUCT: optional string
15: L_SHIPMODE: optional string
16: L_COMMENT: optional string
}
do not match.
```
It appears snowflake writes parquet files with different schema when using
iceberg tables. Should we support this scenario?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
