CTTY commented on code in PR #3670:
URL: https://github.com/apache/iceberg-rust/pull/3670#discussion_r4592960753


##########
crates/iceberg/src/spec/manifest/reader.rs:
##########
@@ -51,8 +55,15 @@ impl ManifestReader {
             None => input_file.read().await?,
         };
 
-        let (metadata, mut entries) =
-            Manifest::try_from_avro_bytes(&bytes, 
Some(&manifest_file.manifest_path))?;
+        let location = manifest_file.manifest_path.clone();
+        let (metadata, mut entries) = self
+            .runtime
+            .cpu()
+            .spawn(async move { Manifest::try_from_avro_bytes(&bytes, 
Some(&location)) })
+            .await
+            // A `JoinError` (the parse panicked, or the cpu runtime shut down)
+            // names no manifest; a parse error already does.
+            .map_err(|err| err.with_context("manifest", 
&manifest_file.manifest_path))??;
 
         for entry in &mut entries {
             entry.inherit_data(manifest_file);

Review Comment:
   we only offload Manifest::try_from_avro_bytes to the CPU runtime, while 
inherit_data and assign_first_row_ids still run on the calling task 
(potentially an IO worker during scan planning)
   
   Worth doing more research here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to