raghuvanshraj commented on issue #2546: URL: https://github.com/apache/iceberg-rust/issues/2546#issuecomment-6852015215
I'd like to take this. Now that #2880 reads unshredded variants, I'll make the Parquet writer accept a variant column (today `parquet_writer.rs` returns `FeatureUnsupported`) and write it as the `metadata`/`value` group with the `VARIANT` annotation, by enabling `variant_experimental` on the parquet dependency and keeping the `arrow.parquet.variant` tag on the field. One thing to flag: parquet 60.1 writes the annotation without `specification_version`, and Spark 4.1 rejects such files. arrow-rs main fixes that (apache/arrow-rs#11961) but it is not released yet, so the PR will need the next arrow-rs release or an assertion that fails until then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
