sdd commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2820378745
I agree with @liurenjie1024 around caching in the FileIO layer. My
experience of trying this was that the lack of context about what the type of
each file is by the time you are ins
liurenjie1024 commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2815436554
I think it's reasonable to have object cache in `iceberg` crate, since
planning is the core functionality of this library, which will be used by all
kinds of users.
But
Xuanwo commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2809902851
> I think adding dynamic cache layer is one option, but not ideal. I think a
better chioce maybe putting caching layer in other components like data file
reader, which could do f
liurenjie1024 commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2809502839
> I think cache should not be done inside iceberg crate, but we shall add
layer function to inject the dynamic cache layer in FileIO struct.
I think adding dynamic c
sundy-li commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2809425682
I think cache should not be inside iceberg crate, but we shall add layer
function to inject the dynamic cache layer in FileIO struct.
--
This is an automated message
liurenjie1024 commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2809399003
Thanks @sundy-li for raising this. I have concerns adding caching
functionality into `FileIO`. Caching could be a complex problem, which may
involve memory management, cac
Xuanwo commented on issue #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217#issuecomment-2808591416
Thank you so much for getting this started! As I mentioned in
https://github.com/apache/iceberg-rust/issues/1036, I believe iceberg-rust
should offer a mechanism that allows user
sundy-li opened a new issue, #1217:
URL: https://github.com/apache/iceberg-rust/issues/1217
### Is your feature request related to a problem or challenge?
Currently, FileIO will use `create_operator` to construct an operator reader
to fetch data from storage.
But when reading t
