[
https://issues.apache.org/jira/browse/KAFKA-20230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Wittwer updated KAFKA-20230:
------------------------------------
Summary: Allow globalTable to provide an optional KeyValueMapper to
materialize an alternative K,V (was: globaltable processor)
> Allow globalTable to provide an optional KeyValueMapper to materialize an
> alternative K,V
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-20230
> URL: https://issues.apache.org/jira/browse/KAFKA-20230
> Project: Kafka
> Issue Type: Improvement
> Reporter: Brandon Wittwer
> Priority: Major
>
> Prior to KAFKA-7663, developers were required to maintain a separate stream
> processor or latent sub-topology to populate a source topic to be used by
> addGlobalStore because the Processor provided in addGlobalStore was not used
> during the restore phase, instead projecting the raw topic into memory.
> Kafka-7663 made it such that the supplied Processor is reliably used not only
> on initial ingestion, but also on the restore. This gives the benefit of
> being able to re-key into a global state store for use in process() steps.
> The DSL method globalTable() now lags behind addGlobalStore in this respect.
>
> Proposal:
> Allow for an overload of globalTable(...), that accepts an optional
> KeyValueMapper to be provided which would override the ProcessorSupplier that
> is currently provided as a TableSource
> [here.|https://github.com/apache/kafka/blob/c4631a222fb054e238af445884fb7bbc9be68970/streams/src/main/java/org/apache/kafka/streams/kstream/internals/InternalStreamsBuilder.java#L191C1-L193C123]
> I don't have the skill or knowledge of kafka internals well enough to make
> a suggestion for how to do this.
> Further Justification:
> Our teams have created unfortunate workarounds for this scenario which
> requires either multiple independent stream processors to be created so that
> the mutations required on <K,V> are
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
