RussellSpitzer commented on PR #11895: URL: https://github.com/apache/iceberg/pull/11895#issuecomment-2568194825
I wonder if this is as important if we switch ParallelIterable to use the implementation suggested here https://github.com/apache/iceberg/issues/11768 which limits the queue depth significantly and changes the yielding behavior. I think it's a good perf change here but I do worry about disconnecting the poll/push operations from actually changing the size tracker for the queue. We probably aren't actually going to have any issues here though since we are already check the size as basically random times without regard to ongoing concurrent operations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
