ajantha-bhat opened a new issue, #8176: URL: https://github.com/apache/iceberg/issues/8176
### Query engine _No response_ ### Question Why the logical types are handled differently between Iceberg-Avro and Iceberg-Parquet? Iceberg Schema of type Date will be translated to an Avro Schema of Int with logical type as Date. Same will be translated to a Parquet schema of Int32 with logical type as Date. But [Avro writer uses Int writer](https://github.com/apache/iceberg/blob/5cad65ad1fbbf214675de6a19f068e5672cb25a7/core/src/main/java/org/apache/iceberg/avro/GenericAvroWriter.java#L102) and expects the data to be an Integer. Where as [Parquet writer uses Date writer](https://github.com/apache/iceberg/blob/5cad65ad1fbbf214675de6a19f068e5672cb25a7/parquet/src/main/java/org/apache/iceberg/data/parquet/BaseParquetWriter.java#L195) and expects data to be LocalDate. **More info:** Assume an Iceberg table partitioned by a date column. When DATE '2023-01-02' is inserted, manifest stores the partitionData for this as 19359 (integer representation of the same date). I am planning to write this partition value into Avro/parquet file (yeah, I am working on partition stats POC). With the same Iceberg schema (date type) and the partitionData (19359), Avro can write success where as Parquet gives me the class cast exception (it expects LocalDate instead of Int). Avro and Parquet Writer code used in a gist https://gist.github.com/ajantha-bhat/0036ceca951f7355c427dac1bc994de9 **Is there any other Parquet API I should use for this scenario?** I have an Integer data and want to insert into a parquet column of type date (Iceberg data type). Note: Above is just by playing with API's from iceberg-parquet module. Via spark, again Parquet writing has no problem. Because [Date type is mapped to integer here](https://github.com/apache/iceberg/blob/5cad65ad1fbbf214675de6a19f068e5672cb25a7/spark/v3.1/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetWriters.java#L149-L153). (and completely ignore logical types) So, is it a bug that is missed to handle the logical types correctly in iceberg-parquet module? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
