RussellSpitzer commented on code in PR #11781:
URL: https://github.com/apache/iceberg/pull/11781#discussion_r1901095259
##########
core/src/main/java/org/apache/iceberg/util/ParallelIterable.java:
##########
@@ -257,17 +263,21 @@ private static class Task<T> implements
Supplier<Optional<Task<T>>>, Closeable {
@Override
public Optional<Task<T>> get() {
try {
+ if (queue.size() >= approximateMaxQueueSize) {
+ // Tasks might hold references (via iterator) to constrained
resources
Review Comment:
Checking the code, this comment seems to be in conversation with the
previous comment that was removed below, it also removes the context of the
"queue / ApproximateMaxQueueSize" check.
I think it should definitely mention first that we "yield when the queue is
over the size limit". The rest of this information isn't really relevant to
this block of code without knowing that it previously used to allow for
yielding in between elements of the iterator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org
For additional commands, e-mail: issues-h...@iceberg.apache.org
