Re: [I] Make FileIO a Trait [iceberg-rust]

2025-06-05 Thread via GitHub
linhr commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2947808661 > Like what we do for `Catalog`, maybe we can add `Storage` trait and `FileIO` struct in the core and have something like `iceberg-storage-opendal` and `iceberg-storage-object-sto

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-29 Thread via GitHub
Xuanwo commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2919079557 Sorry for I didn't join the party earlier. --- I agree with the idea of making `Storage` a trait and turning `FileIO` into a wrapper that can be used throughout the

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-29 Thread via GitHub
liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2918961357 > I'm not sure if I understand this though, especially the difference between FileIO and DynFileIO. Also it seems InputFile etc. are trait now, and I'm worried that it may

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-25 Thread via GitHub
linhr commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2908438919 A good `FileIO`/`Storage` abstraction not only benefits object_store integration, but also makes it easier to work with OpenDAL in Iceberg. For example, I noticed that ther

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-24 Thread via GitHub
tustvold commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2906690237 > Also another concern is that, if we delegate a core abstraction like FileIO to object_store, we may experience unnecessary breaking changes introduces when object_store evolv

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-23 Thread via GitHub
roeap commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2903969557 Just sharing some experiences from the delta world which may not immediately applicable to the question around which trait to use, but maybe be food for thought as to where things

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-22 Thread via GitHub
linhr commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2903122746 > That's the problem, we are defining a richer interface that's not used by iceberg, then it would be confusing for people who want to define their own FileIO implementation, whic

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-21 Thread via GitHub
liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2900075321 > > For example, it requires copy/list as required methods, which are not required by FileIO(this interface is optional). > > You can and people do leave methods uni

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-20 Thread via GitHub
tustvold commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2893312272 > Since the goal of this refactoring is to make underlying implementation extensible and allowing user to choose provider freely, I don't think it's a good idea to bind the int

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-20 Thread via GitHub
liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2893204214 > > I still recommend wrapping it in a struct when passing around in the crate > So are you proposing making Storage a trait instead? If so for the sake of argument

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-19 Thread via GitHub
tustvold commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2893032985 > I still recommend wrapping it in a struct when passing around in the crate So are you proposing making Storage a trait instead? If so for the sake of argument, could th

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-19 Thread via GitHub
liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2892787039 Despite the discussion points raised by @tustvold , I have other things to discuss: 1. Should we have an unified `FileIOBuilder` trait, just like what we did for `Catal

Re: [I] Make FileIO a Trait [iceberg-rust]

2025-05-19 Thread via GitHub
liurenjie1024 commented on issue #1314: URL: https://github.com/apache/iceberg-rust/issues/1314#issuecomment-2892773303 Hi, sorry for being late for this party, and thanks @tustvold for the summary of the discussions. > 1. Should implementations pass around Archttps://github.com/apach