gortiz commented on code in PR #11496:
URL: https://github.com/apache/pinot/pull/11496#discussion_r1318700209
##########
pinot-core/src/main/java/org/apache/pinot/core/transport/DataTableHandler.java:
##########
@@ -83,5 +98,12 @@ protected void channelRead0(ChannelHandlerContext ctx,
ByteBuf msg) {
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
LOGGER.error("Caught exception while handling response from server: {}",
_serverRoutingInstance, cause);
_brokerMetrics.addMeteredGlobalValue(BrokerMeter.RESPONSE_FETCH_EXCEPTIONS, 1);
+ if (cause instanceof java.lang.OutOfMemoryError) {
Review Comment:
Usually is not a good idea to try to recover from a OOM error.
First, they can are thrown anyway, not just in the place where most of the
memory is being consumed.
Second, they fail in random spots and therefore may leave the system in an
unexpected state.
Third, while we are catching the exception we may be in a situation where we
are actually out of memory, so the code that catches the exception may also
throw a new OOM. In this case, for example, we are logging and formatting
strings (which generates garbage) and calling `markServerDown`, which may
allocate.
Usually it is just better to turn off the system instead of trying to
recover.
Assuming we still want to catch the exception: that is the reason why we
cannot remove the element from the map here?
Also: Shouldn't we just drop the messages in `channelRead0` when
`responseSize` is larger than a safe value?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
