jasperjiaguo commented on code in PR #11496:
URL: https://github.com/apache/pinot/pull/11496#discussion_r1320188540
##########
pinot-core/src/main/java/org/apache/pinot/core/transport/DataTableHandler.java:
##########
@@ -83,5 +98,12 @@ protected void channelRead0(ChannelHandlerContext ctx,
ByteBuf msg) {
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
LOGGER.error("Caught exception while handling response from server: {}",
_serverRoutingInstance, cause);
_brokerMetrics.addMeteredGlobalValue(BrokerMeter.RESPONSE_FETCH_EXCEPTIONS, 1);
+ if (cause instanceof java.lang.OutOfMemoryError) {
Review Comment:
>I'm fine with that, but as said earlier, please apply that catch in a
specific channel so we can keep the separation of concerns. The ability to
protect the system against direct OOM in Netty should not be in the handler
that transforms a ByteBuf into a DataTable.
Moved to a seperate handler.
>BTW, at medium term it would be ideal to implement that transformation in a
streaming fashion. By doing that wouldn't need to create the large buffer in
the first term.
Agreed on streaming fashion. Will also make heap memory accounting/managing
easier on broker side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
