rmannibucau commented on issue #13358: URL: https://github.com/apache/iceberg/issues/13358#issuecomment-3559806643
Hi, Apache Spark brings parquet 1.15 and Apache Iceberg relies on parquet 1.16 API which leads to issues when running some code and in particular maintenance operations. Any plan to ensure it is aligned (not sure if using reflection to be backward compat makes sense) Here is an example failure: ``` NoSuchMethodError LogicalTypeAnnotation$VariantLogicalTypeAnnotation variantType(byte) <- ParquetWithSparkSchemaVisitor.visit(:158) <- V2TableWriteExec ``` Using user classpath first can be a workaround in some cases but has also other nasty side effect so being to run with another version of parquet can be neat at iceberg level IMHO. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
