jhump opened a new issue, #416:
URL: https://github.com/apache/iceberg-go/issues/416

   ### Apache Iceberg version
   
   None
   
   ### Please describe the bug 🐞
   
   According to the [spec](https://iceberg.apache.org/spec/#manifest-lists), 
there is no requirement for such a metadata key in a manifest list, only in a 
manifest file. IIUC, this is because migrating a table from v1 to v2 involves 
re-writing all of the manifest lists to v2, but leaving the manifest and data 
files alone. So the version for a manifest list file actually comes from the 
table metadata, not from inside the manifest list file.
   
   I assume the Iceberg Java implementation stores a "format-version" metadata 
key in manifests lists, too, or else this behavior would have been discovered 
to be problematic in tests and by users long before now.
   
   So I think this library should probably be lenient and allow the caller to 
provide a version (that comes from the metadata JSON file), and it can use that 
if there is no "format-version" metadata key (and it can also return an error 
if it does contain a "format-version" that disagrees with the provided 
version). That would make it more spec-adherent, and by that I mean able to 
support tables created by tools/libraries other than iceberg-java that are 
technically following the Iceberg spec.
   
   (I'm not filing this because I've encountered a real issue in practice yet. 
I noticed this discrepancy when ramping up on the Iceberg specification, 
learning how to use this module, and mapping things between the two to better 
understand it.)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org

Reply via email to