sdd commented on PR #373: URL: https://github.com/apache/iceberg-rust/pull/373#issuecomment-2111009920
I've updated this to ditch the concurrency when processing `ManifestEntry` items within a single `Manifest`, producing them asynchronously but sequentially instead. I've kept the limited concurrency when processing `ManifestFile`s within the scan's snapshot's `ManifestList`. I've kept the approach of using an mpsc channel with a spawned task, with that task using `try_for_each_concurrent` to achieve the concurrency. This is because without the channel and spawned task, we'd need to use an async closure, which is unstable rust. With the spawned task we only need to use an async block, which is in stable rust. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
