tustvold commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2893312272
> Since the goal of this refactoring is to make underlying implementation extensible and allowing user to choose provider freely, I don't think it's a good idea to bind the interface to a specific vendor. This _is_ the goal of ObjectStore, it represents an abstraction that has been developed and iterated on by the Rust data ecosystem to allow decoupling things like DataFusion and polars from specific implementations. Now I'm not claiming it is perfect, but it does represent a non-trivial accumulation of knowledge up to this point. As with anything there are engineering tradeoffs, and it is possible iceberg has different requirements, and that's fine. But my hope is that by finding out what these are: * We can ensure the proposed design actually delivers these requirements * We can potentially take some learnings back to ObjectStore to benefit the ecosystem more broadly * We avoid NotInventedHere syndrome > For example, we also want to allow user to use OpenDAL. There is object_store_opendal > For example, it requires copy/list as required methods, which are not required by FileIO(this interface is optional). You can and people do leave methods unimplemented. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
