Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-23 Thread via GitHub
liurenjie1024 merged PR #806: URL: https://github.com/apache/iceberg-rust/pull/806 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@ic

Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-20 Thread via GitHub
sdd commented on code in PR #806: URL: https://github.com/apache/iceberg-rust/pull/806#discussion_r1894298314 ## crates/iceberg/src/arrow/reader.rs: ## @@ -130,62 +128,41 @@ pub struct ArrowReader { impl ArrowReader { /// Take a stream of FileScanTasks and reads all the fi

Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-20 Thread via GitHub
sdd commented on code in PR #806: URL: https://github.com/apache/iceberg-rust/pull/806#discussion_r1894298314 ## crates/iceberg/src/arrow/reader.rs: ## @@ -130,62 +128,41 @@ pub struct ArrowReader { impl ArrowReader { /// Take a stream of FileScanTasks and reads all the fi

Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-20 Thread via GitHub
sdd commented on PR #806: URL: https://github.com/apache/iceberg-rust/pull/806#issuecomment-2557564776 > Hi, try_buffer_unordered does execute concurrently that multiple futures are been polled at the same time by multiple async runtime worker threads. In that case, great - `try_buffe

Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-20 Thread via GitHub
Xuanwo commented on PR #806: URL: https://github.com/apache/iceberg-rust/pull/806#issuecomment-2556576097 > I'm not sure this is what we want. `try_buffer_unordered` executes futures **concurrently** according to the docs: [docs.rs/futures/latest/futures/stream/trait.TryStreamExt.html#metho

Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-20 Thread via GitHub
sdd commented on PR #806: URL: https://github.com/apache/iceberg-rust/pull/806#issuecomment-2556556568 I'm not sure this is what we want. `try_buffer_unordered` executes futures **concurrently** according to the docs: https://docs.rs/futures/latest/futures/stream/trait.TryStreamExt.html#met

Re: [PR] refactor: Remove spawn and channel inside arrow reader [iceberg-rust]

2024-12-15 Thread via GitHub
Xuanwo commented on PR #806: URL: https://github.com/apache/iceberg-rust/pull/806#issuecomment-2544837741 Also cc @sdd for a look, thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s