61yao commented on code in PR #10099:
URL: https://github.com/apache/pinot/pull/10099#discussion_r1072499968
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/MailboxContentStreamObserver.java:
##########
@@ -75,7 +98,9 @@ public MailboxContentStreamObserver(GrpcMailboxService
mailboxService,
StreamObserver<Mailbox.MailboxStatus> responseObserver, boolean
isEnabledFeedback) {
_mailboxService = mailboxService;
_responseObserver = responseObserver;
- _receivingBuffer = new
ArrayBlockingQueue<>(DEFAULT_MAILBOX_QUEUE_CAPACITY);
+ // TODO: Replace unbounded queue with bounded queue when we have
backpressure in place.
+ // It is possible this will create high memory pressure since we have
memory leak issues.
+ _receivingBuffer = new LinkedBlockingQueue();
Review Comment:
This doesn't make it worse because it is always this way. We were simply
dropping data block and returning the wrong result. The memory situation is
always there. Also the in memory mailbox is using the same work around today.
We have to have back pressure in place to fix this issue.
##########
pinot-query-runtime/src/main/java/org/apache/pinot/query/mailbox/channel/MailboxContentStreamObserver.java:
##########
@@ -59,13 +63,32 @@ private static Mailbox.MailboxContent
createErrorContent(Throwable e)
private final boolean _isEnabledFeedback;
private final AtomicBoolean _isCompleted = new AtomicBoolean(false);
- private final ArrayBlockingQueue<Mailbox.MailboxContent> _receivingBuffer;
-
- ReadWriteLock _errorLock = new ReentrantReadWriteLock();
- private Mailbox.MailboxContent _errorContent = null; // Guarded by
_errorLock.
+ private final BlockingQueue<Mailbox.MailboxContent> _receivingBuffer;
+
+ private ReadWriteLock _bufferSizeLock = new ReentrantReadWriteLock();
+ @GuardedBy("bufferSizeLock")
+ private int _maxBufferSize = 0;
Review Comment:
yeah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
