Hello Kafka Community,
We have a use case that requires attaching custom metadata to topics —
things like SLA, durability guarantees, and PagerDuty account references.
While exploring this in Kafka 4.2.0, we noticed an interesting
inconsistency between broker-level and topic-level configuration behavior.
Broker-level custom configs work fine:
Running the following command succeeds without errors:
./bin/kafka-configs.sh --bootstrap-server localhost:9092 \
--entity-type brokers --entity-name 1 \
--alter --add-config 'durability.level=1'
And describing the broker returns the config as expected:
./bin/kafka-configs.sh --bootstrap-server localhost:9092 \
--entity-type brokers --entity-name 1 \
--describe
Output:
Dynamic configs for broker 1 are:
durability.level=null sensitive=true
synonyms={DYNAMIC_BROKER_CONFIG:durability.level=null}
Topic-level custom configs throw an exception:
Running a similar command on a topic:
./bin/kafka-configs.sh --bootstrap-server localhost:9092 \
--entity-type topics --entity-name test-topic-metadata \
--alter --add-config 'durability.level=3'
...results in an InvalidConfigurationException.
Our question:
Is this asymmetry intentional? Is there a design reason why unknown/custom
configurations are permitted at the broker level but rejected at the topic
level? And is there a recommended approach for attaching arbitrary custom
metadata to topics in Kafka?
Any guidance or pointers to relevant KIPs would be greatly appreciated!
Thanks in advance [image: :pray:]
