laskoviymishka commented on code in PR #3670:
URL: https://github.com/apache/iceberg-rust/pull/3670#discussion_r4611046736


##########
crates/iceberg/src/table.rs:
##########
@@ -318,7 +322,7 @@ impl Table {
 
     /// Creates a [`ManifestReader`] for loading manifests referenced by this 
table.
     pub fn manifest_reader(&self) -> ManifestReader {
-        ManifestReader::new(self.file_io.clone())
+        ManifestReader::new(self.file_io.clone(), self.runtime.clone())

Review Comment:
   Could the rustdoc mention that `read` now parses on the table's cpu runtime? 
On `main` it needed no runtime. Here it returns `Unexpected` once that runtime 
is shut down, and `drop_table_data` picks this up too 
([utils.rs](https://github.com/apache/iceberg-rust/blob/780a42810b7ca37fb17731f65ef87e2fb33901e1/crates/iceberg/src/catalog/utils.rs#L147-L163)).
 That fits the [`Runtime` 
contract](https://github.com/apache/iceberg-rust/blob/780a42810b7ca37fb17731f65ef87e2fb33901e1/crates/iceberg/src/runtime/mod.rs#L127-L132),
 so one doc line is enough.



##########
crates/iceberg/src/scan/mod.rs:
##########
@@ -3620,6 +3620,92 @@ mod tests {
         assert!(result.is_ok(), "Scan timed out - deadlock detected");
     }
 
+    /// Plans `table` under a timeout so a swallowed manifest error shows up as
+    /// a timeout instead of a hang, and returns the collected result.
+    async fn plan_files_with_timeout(table: &Table) -> 
Result<Vec<FileScanTask>> {
+        tokio::time::timeout(std::time::Duration::from_secs(5), async {
+            table
+                .scan()
+                .build()
+                .unwrap()
+                .plan_files()
+                .await
+                .unwrap()
+                .try_collect::<Vec<_>>()
+                .await
+        })
+        .await
+        .expect("plan_files hung instead of reporting the manifest error")
+    }
+
+    /// Returns the path of the first manifest in the current snapshot.
+    async fn first_manifest_path(fixture: &TableTestFixture) -> String {
+        let snapshot = fixture.table.metadata().current_snapshot().unwrap();
+        let manifest_list = fixture
+            .table
+            .manifest_list_reader(snapshot)
+            .load()
+            .await
+            .unwrap();
+        manifest_list.entries()[0].manifest_path.clone()
+    }
+
+    #[tokio::test]
+    async fn test_plan_files_reports_corrupt_manifest_as_error() {
+        let mut fixture = TableTestFixture::new();
+        fixture.setup_manifest_files().await;
+        let path = first_manifest_path(&fixture).await;
+        fs::write(&path, b"not an avro file").unwrap();
+
+        let err = plan_files_with_timeout(&fixture.table)
+            .await
+            .expect_err("a corrupt manifest must fail the plan");
+        assert_eq!(err.kind(), ErrorKind::Unexpected);
+        assert!(err.to_string().contains(&path), "{err}");
+    }
+
+    #[tokio::test]
+    async fn test_plan_files_reports_missing_manifest_as_error() {
+        let mut fixture = TableTestFixture::new();
+        fixture.setup_manifest_files().await;
+        let path = first_manifest_path(&fixture).await;
+        fs::remove_file(&path).unwrap();
+
+        let err = plan_files_with_timeout(&fixture.table)
+            .await
+            .expect_err("a missing manifest must fail the plan");
+        assert_eq!(err.kind(), ErrorKind::Unexpected);
+        assert!(err.to_string().contains(&path), "{err}");
+    }
+
+    /// A manifest parse spawned on a shut-down cpu runtime is cancelled, which
+    /// takes the same `JoinError` path a panicking parse does. The error must
+    /// still name the manifest, which a `JoinError` on its own does not.

Review Comment:
   Should the last sentence credit `ObjectCache`? As I read it, with the 
default cache the path comes from its "Failed to load manifest {path}" wrapper 
([object_cache.rs](https://github.com/apache/iceberg-rust/blob/780a42810b7ca37fb17731f65ef87e2fb33901e1/crates/iceberg/src/io/object_cache.rs#L166-L177)),
 so this assertion holds even without the `with_context` in the reader. 
`test_read_names_manifest_when_parse_is_cancelled` is the test that pins the 
reader's own context.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to