[
https://issues.apache.org/jira/browse/KAFKA-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620486#comment-16620486
]
Shantanu Deshmukh commented on KAFKA-7422:
------------------------------------------
What do you mean by valid SSL store and valid SSL port?
> Huge memory consumption (leak?) by Kafka Producer
> -------------------------------------------------
>
> Key: KAFKA-7422
> URL: https://issues.apache.org/jira/browse/KAFKA-7422
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.10.1.0
> Environment: OS - AIX 7.1, Java - IBM Java 1.8
> Reporter: Shantanu Deshmukh
> Priority: Major
> Attachments: heapdump.20180918.111423.4587876.0002.phd
>
>
> Hello,
>
> We have a 3 broker Kafka 0.10.1.0 deployment in production. There are some
> applications which have Kafka Producers embedded in them which send
> application logs to a topic. This topic has 10 partitions with replication
> factor of 3.
>
> We are observing that memory usage on some of these application servers keep
> shooting through the roof intermittently. After taking heapdump we found out
> that top suspects were:
> *---------------------*
> *org.apache.kafka.common.network.Selector -*
> occupies *352,519,104 (24.96%)* bytes. The memory is accumulated in one
> instance of *"byte[]"* loaded by *"<system class loader>"*.
> *org.apache.kafka.common.network.KafkaChannel -*
> occupies *352,527,424 (24.96%)* bytes. The memory is accumulated in one
> instance of *"byte[]"* loaded by *"<system class loader>"*
> *---------------------*
> Both of these were holding about 352MB of space. 3 such instances, so they
> were consuming about 1.2GB of memory.
> Now regarding usage of producers. Not a huge amount of logs are being sent to
> Kafka cluster. It is about 200 msgs/sec. Only one producer object is being
> used throughout application. Async send function is used.
> What could be the cause of such huge memory usage? Is this some sort of
> memory leak in this specific Kafka version?
> Here's producer config being used at the present.
> {{>>>>>>>>>>>>>>>>}}
> {{kafka.bootstrap.servers=x.x.x.x:9092,x.x.x.x:9092,x.x.x.x:9092}}
> {{ kafka.acks=0}}
> {{
> kafka.key.serializer=org.apache.kafka.common.serialization.StringSerializer}}
> {{
> kafka.value.serializer=org.apache.kafka.common.serialization.StringSerializer}}
> {{[kafka.max.block.ms|http://kafka.max.block.ms/]=1000}}
> {{[kafka.request.timeout.ms|http://kafka.request.timeout.ms/]=1000}}
> {{ kafka.max.in.flight.requests.per.connection=1}}
> {{ kafka.retries=0}}
> {{ kafka.compression.type=gzip}}
> {{ kafka.security.protocol=SSL}}
> {{ kafka.ssl.truststore.location=/data/kafka/kafka-server-truststore.jks}}
> {{ kafka.ssl.truststore.password=XXXXXX}}
> {{ logger.level=INFO }}{{<<<<<<<<<<<<<<}}
> Attaching heapdump for your perusal.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
