DieHertz commented on issue #1229: URL: https://github.com/apache/iceberg-python/issues/1229#issuecomment-2414395991
> There's already an ExecutorFactory, do you think we can use that instead of ProcessPoolExecutor? The issue with the `ExecutorFactory` is it's using a `ThreadPoolExexutor` instead of a process pool, and the operation we're trying to parallelize doesn't play well with GIL in one process. At least I'm not seeing such a significant profit when using it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
