[
https://issues.apache.org/jira/browse/KAFKA-19519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lan Ding updated KAFKA-19519:
-----------------------------
Description:
|In KAFKA-19427, there's a use case that when there is a consumer group
subscribes huge amount of topics/partitions, when this group is rebalanced, and
then the coordinator broker stores the assignment of this group to
__consumer_offsets, it will throw an error RecordTooLargeException .
Currently, to resolve this issue, the only solution is to increase the
broker-level `message.max.bytes` config. But the side effect of this change is
it potentially allows all topics without override the topic level
\{{max.message.bytes}} config, will now allow higher message size.
We could introduce a new config to drive the value used by the group
coordinator - e.g group.coordinator.append.max.bytes - instead of relying on
the broker message.max.bytes. This would be used to set the max bytes at the
topic level when the topic is created.|
was:
在 KAFKA-19427
中，有一个用例是，当有一个消费者组订阅了大量的主题/分区时，当该组被重新平衡时，然后协调器代理将该组的分配存储给__consumer_offsets，它将抛出错误
RecordTooLargeException 。
目前，要解决这个问题，唯一的解决方案是增加代理级别的“message.max.bytes”配置。但此更改的副作用是它可能允许所有主题而不覆盖主题级别
{{max.message.bytes}} 配置，现在将允许更高的消息大小。
我们可以引入一个新的配置来驱动组协调器使用的值（例如 group.coordinator.append.max.bytes），而不是依赖代理
message.max.bytes。这将用于在创建主题时在主题级别设置最大字节数。
> Introduce a new config for group coordinator max record size
> ------------------------------------------------------------
>
> Key: KAFKA-19519
> URL: https://issues.apache.org/jira/browse/KAFKA-19519
> Project: Kafka
> Issue Type: Improvement
> Reporter: Luke Chen
> Priority: Major
>
> |In KAFKA-19427, there's a use case that when there is a consumer group
> subscribes huge amount of topics/partitions, when this group is rebalanced,
> and then the coordinator broker stores the assignment of this group to
> __consumer_offsets, it will throw an error RecordTooLargeException .
>
>
>
> Currently, to resolve this issue, the only solution is to increase the
> broker-level `message.max.bytes` config. But the side effect of this change
> is it potentially allows all topics without override the topic level
> \{{max.message.bytes}} config, will now allow higher message size.
>
>
>
> We could introduce a new config to drive the value used by the group
> coordinator - e.g group.coordinator.append.max.bytes - instead of relying on
> the broker message.max.bytes. This would be used to set the max bytes at the
> topic level when the topic is created.|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
