dmeaux commented on issue #10260: URL: https://github.com/apache/iceberg/issues/10260#issuecomment-2411589147
Would it be worth it to add internal mechanisms to Iceberg to treat geospatial metadata and coordinate reference system data similar to manifest lists and manifest files? So, an Iceberg JSON metadata file would have key-value structures similar to the "snapshots" key that point to a geospatial metadata list (AVRO) file, which points to one or more geospatial metadata files. Similar to the data snapshot files, these only get created when there is a change (add, update, delete) to the metadata, including the CRS, associated with a geospatial entity in the data. Otherwise the metadata file points to the most current version of the metadata/CRS. We could use ISO IEC 13249 (SQL/MM) and/or ISO 19115 to define the structure of the metadata in the AVRO files. I would think this would minimize "bloat" from including the geospatial metadata as one or more columns directly in the Iceberg JSON metadata file and would also give us more precision than just defining one set of metadata/CRS information for the entire the table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
