[
https://issues.apache.org/jira/browse/KAFKA-17632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
thanhlv updated KAFKA-17632:
----------------------------
Description:
Our project has some special logic that requires custom partitions.
With an odd number of Partitions, everything works fine and well.
However, with an even number of partitions. Data will only be written to
even-numbered Partition IDs
Info:
Lib Java: `kafka-clients:3.8.0`
Code demo:
{code:java}
public class CustomLogicPartitionMain {
public static void main(String[] args) throws IOException {
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "ALL");
final var props = new Properties();
props.setProperty(ProducerConfig.CLIENT_ID_CONFIG,
"java-producer-producerRecordPartition-KeyNotNull");
props.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
"localhost:29091,localhost:29092,localhost:29093,localhost:29094");
props.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
StringSerializer.class.getName());
props.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
StringSerializer.class.getName());
props.setProperty(ProducerConfig.BATCH_SIZE_CONFIG, "5000");
props.setProperty(ProducerConfig.PARTITIONER_CLASS_CONFIG,
"org.apache.kafka.clients.producer.RoundRobinPartitioner");
try (var producer = new KafkaProducer<Object, String>(props)) {
final var messageProducerRecord = new ProducerRecord<>(
"topic-rep-1-partition-10", //topic name
// 36 byte
UUID.randomUUID().toString() // value
);
for (int i = 1; i <= 5000; i++) {
producer.send(messageProducerRecord);
}
}
}
}
{code}
!image-2024-09-27-15-05-53-707.png!
was:
Our project has some special logic that requires custom partitions.
With an odd number of Partitions, everything works fine and well.
However, with an even number of partitions. Data will only be written to
even-numbered Partition IDs
Info:
Lib Java: `kafka-clients:3.8.0`
Code demo:
{code:java}
public class CustomLogicPartitionMain {
public static void main(String[] args) throws IOException {
System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "ALL");
final var props = new Properties();
props.setProperty(ProducerConfig.CLIENT_ID_CONFIG,
"java-producer-producerRecordPartition-KeyNotNull");
props.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
"localhost:29091,localhost:29092,localhost:29093,localhost:29094");
props.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
StringSerializer.class.getName());
props.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
StringSerializer.class.getName());
props.setProperty(ProducerConfig.BATCH_SIZE_CONFIG, "5000");
props.setProperty(ProducerConfig.PARTITIONER_CLASS_CONFIG,
"org.apache.kafka.clients.producer.RoundRobinPartitioner");
try (var producer = new KafkaProducer<Object, String>(props)) {
final var messageProducerRecord = new ProducerRecord<>(
"topic-rep-1-partition-10", //topic name
// 36 byte
UUID.randomUUID().toString() // value
);
for (int i = 1; i <= 5000; i++) {
producer.send(messageProducerRecord);
}
}
}
}
{code}
!image-2024-09-27-15-05-53-707.png!
// write the logic to define the partition. Data is the class path to the
ClassCustomLogicPartitioner class
> Custom `partitioner.class` with an even number of partitions always writes to
> even partitions
> ---------------------------------------------------------------------------------------------
>
> Key: KAFKA-17632
> URL: https://issues.apache.org/jira/browse/KAFKA-17632
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 3.8.0
> Reporter: thanhlv
> Priority: Major
> Attachments: image-2024-09-27-15-05-53-707.png
>
>
> Our project has some special logic that requires custom partitions.
> With an odd number of Partitions, everything works fine and well.
> However, with an even number of partitions. Data will only be written to
> even-numbered Partition IDs
> Info:
> Lib Java: `kafka-clients:3.8.0`
> Code demo:
> {code:java}
> public class CustomLogicPartitionMain {
> public static void main(String[] args) throws IOException {
> System.setProperty("org.slf4j.simpleLogger.defaultLogLevel", "ALL");
> final var props = new Properties();
> props.setProperty(ProducerConfig.CLIENT_ID_CONFIG,
> "java-producer-producerRecordPartition-KeyNotNull");
> props.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
> "localhost:29091,localhost:29092,localhost:29093,localhost:29094");
> props.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,
> StringSerializer.class.getName());
> props.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,
> StringSerializer.class.getName());
> props.setProperty(ProducerConfig.BATCH_SIZE_CONFIG, "5000");
> props.setProperty(ProducerConfig.PARTITIONER_CLASS_CONFIG,
> "org.apache.kafka.clients.producer.RoundRobinPartitioner");
> try (var producer = new KafkaProducer<Object, String>(props)) {
> final var messageProducerRecord = new ProducerRecord<>(
> "topic-rep-1-partition-10", //topic name
> // 36 byte
> UUID.randomUUID().toString() // value
> );
> for (int i = 1; i <= 5000; i++) {
> producer.send(messageProducerRecord);
> }
> }
> }
> }
> {code}
> !image-2024-09-27-15-05-53-707.png!
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
