[
https://issues.apache.org/jira/browse/KAFKA-17343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17882835#comment-17882835
]
Eric Chang commented on KAFKA-17343:
------------------------------------
I think It’s caused by laking of container awareness in `openjdk:8`, in
`openjdk:8` , jvm will estimate max heap size based on memory capacity of host
machine:
```
docker run -it --cpus 1 -m 1G openjdk:8 java -XshowSettings:vm -version
VM settings:
Max. Heap Size (Estimated): 3.48G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM
openjdk version "1.8.0_342"
OpenJDK Runtime Environment (build 1.8.0_342-b07)
OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode)
⋊> ~/r/u/k/feat-update-base-image on feat-update-base-image ⨯
```
You can see that even if we limit container memory to 1G, Max. Heap Size
(Estimated) is still 3.48G, exceeded the container memory limit `1G`
and if you add `-verbose:gc -XX:+PrintGCDetails` as `KAFKA_HEAP_OPTS`
you can see memory is keep growing, even exceeded our container memory limit
`2G`
```
[Eden: 22528.0K(22528.0K)->0.0B(37888.0K) Survivors: 4096.0K->4096.0K Heap:
96697.5K(252.0M)->85799.0K(504.0M)]
[Eden: 37888.0K(37888.0K)->0.0B(45056.0K) Survivors: 4096.0K->6144.0K Heap:
120.8M(504.0M)->102.3M(1008.0M)]
[Eden: 45056.0K(45056.0K)->0.0B(74752.0K) Survivors: 6144.0K->7168.0K Heap:
146.3M(1008.0M)->117.4M(1609.0M)]
[Eden: 74752.0K(74752.0K)->0.0B(96256.0K) Survivors: 7168.0K->10240.0K Heap:
190.4M(1609.0M)->142.2M(2090.0M)]
[Eden: 96256.0K(96256.0K)->0.0B(110.0M) Survivors: 10240.0K->13312.0K Heap:
236.2M(2090.0M)->174.5M(2474.0M)]
```
If you use a openjdk with container awareness (8u372)
```
docker run -it --cpus 1 -m 2G phenompeople/openjdk:8u372 java -XshowSettings:vm
-version
VM settings:
Max. Heap Size (Estimated): 494.94M
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM
openjdk version "1.8.0_372"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_372-b07)
OpenJDK 64-Bit Server VM (Temurin)(build 25.372-b07, mixed mode)
```
Max Heap Size will be limited to about 1/4 of container memory limit.
And if we run system test with this jdk:
```diff
./tests/docker/ducker-ak up --jdk "phenompeople/openjdk:8u372"
```
You can see that Heap Size is limited to 1/4 of container limit (512MB)
```bash
[Eden: 21504.0K(21504.0K)->0.0B(21504.0K) Survivors: 4096.0K->4096.0K Heap:
256.1M(512.0M)->232.6M(512.0M)]
[Eden: 21504.0K(21504.0K)->0.0B(21504.0K) Survivors: 4096.0K->4096.0K Heap:
253.6M(512.0M)->239.1M(512.0M)]
[Eden: 21504.0K(21504.0K)->0.0B(21504.0K) Survivors: 4096.0K->4096.0K Heap:
260.1M(512.0M)->242.2M(512.0M)]
```
> Fix failed benchmark_test.py
> ----------------------------
>
> Key: KAFKA-17343
> URL: https://issues.apache.org/jira/browse/KAFKA-17343
> Project: Kafka
> Issue Type: Bug
> Reporter: Chia-Ping Tsai
> Assignee: Chia-Ping Tsai
> Priority: Blocker
> Fix For: 4.0.0
>
> Attachments: producer_performance-oom.log,
> producer_performance-oom.stderr, producer_performance-oom.stdout
>
>
> see discussion:
> https://github.com/apache/kafka/pull/16646#issuecomment-2289176368
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
