[
https://issues.apache.org/jira/browse/KAFKA-17198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868564#comment-17868564
]
Xuze Yang commented on KAFKA-17198:
-----------------------------------
When creating topic, the broker list is unordered, which leads to this issue:
!broker list is unordered.png!
As a comparison, when alter topic, the broker list is ordered:
!broker list is ordered.png!
According to the current partition assignment rule, if the broker lists for
create and alter topic are both ordered (0, 1, 2), there will be no problem,
and the three possible assignment results are all evenly distributed:
|| ||partition 1||partition 2||partition 3||
|case 1|0|1|(2+0)%3=2|
|case 2|1|2|(2+1)%3=0|
|case 3|2|0|(2+2)%3=1|
If the broker list used to create a topic is unordered, such as (0, 2, 1) as
shown in the above figure, problems may arise and the assignmet results may be
uneven， all possible assignment results are：
|| ||partition 1||partition 2||partition 3||
|case 1|0|2|(2+0)%3=2|
|case 2|2|1|(2+2)%3=1|
|case 3|1|0|(2+1)%3=0|
I looked at the code and found that the broker list when creating a topic was
originally intended to be ordered, but in the end it was unordered.
Just sorting the broker list can solve this problem. I have looked at the code
of the trunk branch and it seems that this problem still exists. I will
initiate a PR request later.
Is there anything I missed? Sincerely welcome to discuss.
> Partition imbalance after alter due to inconsistent order of broker list
> ------------------------------------------------------------------------
>
> Key: KAFKA-17198
> URL: https://issues.apache.org/jira/browse/KAFKA-17198
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.7.2
> Reporter: Xuze Yang
> Assignee: Xuze Yang
> Priority: Major
> Attachments: broker list is ordered.png, broker list is
> unordered.png, partion distributed not evenly.png
>
>
> Three broker environments. Create a topic with 1 replica and 2 partition, and
> then alter the topic to 3 partitions, which are not evenly distributed across
> 3 brokers.
> command as belows：
> {code:java}
> #Create a topic with 1 replica and 2 partitions
> kafka-topics.sh --create --bootstrap-server xx.xx.xx.xx:9092
> --replication-factor 1 --partitions 2 --topic test
> #Alter to 3 partitions
> kafka-topics.sh --bootstrap-server xx.xx.xx.xx:9092 --alter --topic test
> --partitions 3
> #View the distribution of topic partitions
> kafka-topics.sh --bootstrap-server xx.xx.xx.xx:9092 --describe --topic test
> {code}
> Repeat the above command 5 times to obtain 5 topics: test1, test2, test3,
> test4, and test5. The partition distribution of the 5 topics is shown below:
> !partion distributed not evenly.png!
> It can be seen that each topic's 3 partitions are only distributed on 2
> brokers, instead of the expected 3 brokers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
