[
https://issues.apache.org/jira/browse/KAFKA-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16151950#comment-16151950
]
Richard Yu commented on KAFKA-5828:
-----------------------------------
In the Fetcher class, there is this {fetchRecord(int maxRecords)} method. It is
indirectly called from the {pollOnce()} method
in ConsumerConfigs. Using the above method, it should be relatively simple to
set a higher bound.
> Allow explicitly setting polling interval for Kafka connectors
> --------------------------------------------------------------
>
> Key: KAFKA-5828
> URL: https://issues.apache.org/jira/browse/KAFKA-5828
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Behrang Saeedzadeh
>
> I have a Kafka cluster deployed in our internal data center and a Kafka
> Connect server deployed in AWS that gets data records from a topic on the
> Kafka cluster and writes them into a Kinesis stream.
> We want to ensure that our Kafka connect server does not saturate the
> available bandwidth between our internal data center and AWS.
> Using {{max.partition.fetch.bytes}} we can limit the upper bound of data that
> can be fetched in each poll call. If we can also configure the poll interval,
> then we can limit how much data is transferred per partition per second.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
