BsoBird commented on issue #10326: URL: https://github.com/apache/iceberg/issues/10326#issuecomment-2133087008
Hi, I've done some work on fixing hadoop_catalog before. In my experience, to use a filesystem-based catalog, you currently need to rely on the filesystem to provide atomic rename operations. Object stores often do not have atomic operations. To use fileSystem_catalog with object storage, you must use some additional middleware to provide atomicity to file system operations. Of course, this is just the status quo. I think there is a lot of work that needs to be done if you want to implement the basic functionality of catalog management on an object store that does not have atomic operations. We can discuss this further if you are interested. But please keep in mind that this is not recommended in the current version. @911432 @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
