nssalian commented on issue #16782: URL: https://github.com/apache/iceberg/issues/16782#issuecomment-4685185095
Thanks for the detailed repro. The class in your stack trace, SparkParquetWriterWithVariantShredding, only existed on the pre-merge state of PR #14297 and was removed before that PR landed on main. The follow up commits to what I can gauge the state of the EMR release has rewired the write path through the FormatModel API and `BufferedFileAppender`, eliminating the visitor that produced this `ClassCastException`). I built a focused repro on apache/main matching your wide schema (20 cols: scalars + 11 VARIANT + 2 ARRAY, v3 table, shred-variants=true, INSERT 2 rows) and it passes - the bug doesn't reproduce on apache/main. @soumilshah1995 please verify and let me know. Let's keep this open for now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
