orthoxerox opened a new issue, #13196: URL: https://github.com/apache/iceberg/issues/13196
### Feature Request / Improvement The DuckLake manifesto (https://ducklake.select/manifesto/) declares that storing the metadata on the blob storage next to the data is suboptimal and we should go back to storing it in the DB backing the catalog. The big three improvements I agree with are: - multi-table transactions are easy, because you can just leverage the transaction guarantees of the metastore DBMS - now that the metadata layer is no longer append-only, you can safely update older versions to point to the rewritten data files, thus preserving time travel - this alleviates the pressure on the blob storage, which might not be happy to manage lots of of tiny metadata files Perhaps it might make sense to add this functionality to the next major revision of the Iceberg catalog spec? ### Query engine None ### Willingness to contribute - [ ] I can contribute this improvement/feature independently - [ ] I would be willing to contribute this improvement/feature with guidance from the Iceberg community - [x] I cannot contribute this improvement/feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
