IgorBerman commented on PR #14744: URL: https://github.com/apache/iceberg/pull/14744#issuecomment-3610513738
@rdblue thanks for the review! The question here what projection schema will look like in a case of `SELECT struct_field, struct_field.sub_field FROM` I've added test that shows that if we use projection with full struct schema then the change will preserve struct_field fully(point in my test) if it will contain only struct_field itself without all the inner fields - then probably it wont work. this depends on how projection is created and defined probably it comes from expectedSchema most probably and the question is how it's defined `this.projection = ParquetSchemaUtil.pruneColumns(fileSchema, expectedSchema);` Maybe we here entering domain of how spark creates expectedSchema. I'd argue that it should contain full struct_field schema. I'll try to do more tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
