lirui-apache commented on PR #7844: URL: https://github.com/apache/iceberg/pull/7844#issuecomment-1862080641
We have been using a blocking queue and shared thread pool for a while. We did hit some "dead lock" issue when running multi-stage queries with trino, because the iceberg split source generates splits synchronously. We fixed it by [making it async](https://github.com/trinodb/trino/issues/17631), and so far everything works fine. I think using an isolated pool would be safer, but it may hurt performance. Since we already have API to specify thread pool for a table scan, how about we let users deicide which pool to use? IMO whether to use blocking queue and whether to use shared pool can be configured orthogonally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
