dhruvarya-db opened a new issue, #3274: URL: https://github.com/apache/iceberg-rust/issues/3274
### Apache Iceberg Rust version main (`0.10.1`, commit 119fad92c) ### Describe the bug If an equality-delete file fails to load (for example, it is missing or fails to parse), its entry in `DeleteFilter` stays in `EqDelState::Loading` forever. Any later read that references the same delete file through the same `DeleteFilter` then waits forever for a wake-up that never comes. How it happens: 1. `CachingDeleteFileLoader` claims the delete file, and `insert_equality_delete` marks it `Loading`. It then spawns a task that awaits a `oneshot` receiver for the parsed predicate. 2. Opening or parsing the file fails, and the loader returns early with `?` before calling `sender.send(...)`, which drops the sender. 3. The spawned task runs `eq_del.await.unwrap()`, gets `RecvError`, and panics. It never sets the entry to `Loaded` and never calls `notify_waiters()`. 4. A later load that uses the same `DeleteFilter` sees the entry as already claimed and skips loading the file. `get_equality_delete_predicate_for_delete_file_path` then waits on the entry's `Notify`, which is never signalled. Clones of an `ArrowReader` share one `CachingDeleteFileLoader` and therefore one `DeleteFilter`. So after one failed read, every other read of that delete file through the reader or any of its clones hangs, instead of returning the error or retrying the load. ### To Reproduce 1. Build an `ArrowReader` and clone it. 2. Read a `FileScanTask` whose equality-delete file can't be read, for example because it doesn't exist. The read returns an error. 3. Read the same task with the clone. The read never completes. ### Expected behavior A failed equality-delete load should not leave a stale `Loading` entry behind. Waiters should be woken and get an error, and a later read should be able to load the file again. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
