siddharthteotia commented on code in PR #11496:
URL: https://github.com/apache/pinot/pull/11496#discussion_r1321750404
##########
pinot-core/src/main/java/org/apache/pinot/core/transport/InstanceRequestHandler.java:
##########
@@ -321,6 +322,10 @@ private void sendResponse(ChannelHandlerContext ctx,
String tableNameWithType, l
LOGGER.info("Slow query: request handler processing time: {}, send
response latency: {}, total time to handle "
+ "request: {}", queryProcessingTimeMs, sendResponseLatencyMs,
totalQueryTimeMs);
}
+ if (serializedDataTable.length > LARGE_RESPONSE_SIZE_THRESHOLD_BYTES) {
+ LOGGER.warn("Large query: response size in bytes: {}, table name {}",
+ serializedDataTable.length, tableNameWithType);
Review Comment:
I think instead of a warning, emitting a metric may be much better to help
us plan future enhancements IMO.
- Understanding why response is so large and what can be done in terms of
optimizing it
- If the server knows that the fan out of query is huge then there is a
chance of having other servers' response size huge as well (and therefore a
chance for causing Direct Memory OOM on broker). In that case, killing it on
the server before responding is better.
> Can we not just kill the query on the server when we detect this ?
I take this back. Simply killing upon detecting response on server is likely
to lead to lot of pre-mature / unnecessary queries getting killed. So may be we
make servers aware of fan-out and then decide to kill.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
