gortiz opened a new pull request, #16939:
URL: https://github.com/apache/pinot/pull/16939
This PR adds 6 new metrics to measure direct memory used by gRPC and Netty:
- On Broker:
- grpcTotalUsedDirectMemory: Total used direct memory used by the version
of Netty shaded by gRPC. It is -1 if grpc-netty is not configured to measure
bytes.
- grpcTotalMaxDirectMemory: Total max direct memory used by the version of
Netty shaded by gRPC.
- On Server:
- grpcTotalUsedDirectMemory: Same as on broker
- grpcTotalMaxDirectMemory: Same as on broker
- nettyTotalUsedDirectMemory: Total used direct memory used by the vanilla
version Netty. It is -1 if Netty is not configured to measure bytes.
- nettyTotalMaxDirectMemory: Total max direct memory used by the vanilla
version Netty.
The metrics prefixed as _netty_ are used by the normal SSE query server,
while the ones prefixed as _grpc_ are used by MSE (for both plan protocol and
mailbox protocol), the gRPC query server and the gRPC query broker.
The following JAVA_OPTS are needed to use Netty correctly:
- --add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
- -Dio.netty.tryReflectionSetAccessible=true
- -Dio.grpc.netty.shaded.io.netty.tryReflectionSetAccessible=true
Without these options, Netty won't be able to manage ByteBuffers optimally
and it won't count (or limit) the amount of direct memory being used.
This PR adds these three options to the bash scripts, as well as to
`.mvn/jvm.config`, a new file Maven reads to set its default JVM options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
