dentiny commented on code in PR #512:
URL: https://github.com/apache/iceberg-rust/pull/512#discussion_r2365940293
##########
crates/iceberg/src/scan.rs:
##########
@@ -477,24 +478,22 @@ impl ManifestFileContext {
/// streaming its constituent [`ManifestEntries`] to the channel provided
in the context
async fn fetch_manifest_and_stream_manifest_entries(self) -> Result<()> {
let ManifestFileContext {
- file_io,
+ object_cache,
manifest_file,
bound_predicates,
snapshot_schema,
field_ids,
- expression_evaluator_cache,
mut sender,
+ expression_evaluator_cache,
..
} = self;
- let file_io_cloned = file_io.clone();
- let manifest = manifest_file.load_manifest(&file_io_cloned).await?;
-
- let (entries, _) = manifest.consume();
+ let manifest = object_cache.get_manifest(&manifest_file).await?;
Review Comment:
I'm actually curious why do we take lookaside cache design, instead of
read-through/write-through cache?
The benefit for later is when you write a new manifest list, you store it in
object cache directly, so next read you don't need to fetch it again (which is
moonlink's usage).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
