[
https://issues.apache.org/jira/browse/KAFKA-17926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17895138#comment-17895138
]
Chia-Ping Tsai commented on KAFKA-17926:
----------------------------------------
{quote}
Could we adjust NUM_BATCHES_TO_RETAIN to match the configured
max.in.flight.requests.per.connection?
{quote}
`max.in.flight.requests.per.connection` is a client-side configuration, while
`NUM_BATCHES_TO_RETAIN` is used by the server. Therefore, each server needs a
way to determine which max.in.flight.requests.per.connection value is being
used by a particular client, since a producer client could send requests to
multiple servers. That means we can't use InitProducerIdRequest to sync
max.in.flight.requests.per.connection with the servers. Instead, we could
enhance the producer request to carry the max.in.flight.requests.per.connection
value, allowing the server to initialize or adjust ProducerStateEntry with the
correct size.
By the way, we should address this in another JIRA, even if the above
discussion is acceptable, since it requires a KIP. It would also be great to
make the idempotent producer the default in version 4.0.
> Improve the documentation explaining why
> max.in.flight.requests.per.connection should not exceed 5.
> ---------------------------------------------------------------------------------------------------
>
> Key: KAFKA-17926
> URL: https://issues.apache.org/jira/browse/KAFKA-17926
> Project: Kafka
> Issue Type: Improvement
> Reporter: Chia-Ping Tsai
> Assignee: PoAn Yang
> Priority: Minor
>
> We should highlight that the server [0] retains record metadata for the 5
> most recent batches, so sending too many requests to a node simultaneously
> can break idempotence.
> [0]
> https://github.com/apache/kafka/blob/af53758746485daa06c4d4366036c04f7f9f0e63/storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateEntry.java#L32
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
