Fokko opened a new pull request, #8672: URL: https://github.com/apache/iceberg/pull/8672
Jan raised a point on Slack of the semantic meaning of a field that can be written: https://apache-iceberg.slack.com/archives/C03LG1D563F/p1695834739711569 There are two options: - The field is not part of the schema, and is omitted from the file - The field is part of the schema, but the value is not written (nullable) My personal take on this is that we should use static schema when writing Avro files so that all the fields that are either optional or required are in the schema. I'm well aware that this doesn't impose any issues if you dogfood the Iceberg Avro reader, where you can add required fields, for example, the `134: content` field in the manifest. However, I think we should try to stick to the concept of writing strictly, read permissive where we try to encourage people to write all the fields that are in the spec (even if the value itself is all null). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
