[
https://issues.apache.org/jira/browse/KAFKA-17744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastien Viale updated KAFKA-17744:
------------------------------------
Description:
The logs for Kafka Streams local state restoration incorrectly refer to the
{{StreamThread}} instead of the {{StateUpdater}} thread, which is responsible
for decoupling the restoration process. The restore consumer group also
references {{StreamThread}} instead of {{{}StateUpdater{}}}, which should be
corrected for clarity.
*Current logs:*
...
stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=2698468, end=5716195,
totalRestored=2698468} \{stateful_app-count-store-name-changelog-0:
position=2655839, end=5743384, totalRestored=2655839}
stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=5415824, end=5716195,
totalRestored=5415824} \{stateful_app-count-store-name-changelog-0:
position=5412953, end=5743384, totalRestored=5412953}
stream-thread [***-StreamThread-1] Finished restoring changelog
stateful_app-count-store-name-changelog-1 to store count-store-name with a
total number of 5716195 records
stream-thread [***-StreamThread-1] Restoration in progress for 1 partitions.
\{stateful_app-count-store-name-changelog-1: position=1000, end=5716195,
totalRestored=1000}
stream-thread [***-StreamThread-1] Finished restoring changelog
stateful_app-count-store-name-changelog-0 to store count-store-name with a
total number of 5743384 records
...
[Consumer clientId=***-StreamThread-1-restore-consumer, groupId=null] Assigned
to partition(s): stateful_app-count-store-name-changelog-1,
stateful_app-count-store-name-changelog-0
...
[Consumer clientId=***-StreamThread-1-restore-consumer, groupId=null] Resetting
offset for partition stateful_app-count-store-name-changelog-0 to position
FetchPosition{offset=0, offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[localhost:19092 (id: 1 rack:
null)], epoch=2}}.
*Expected logs:*
...
...
state-updater [***-StateUpdater-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=2698468, end=5716195,
totalRestored=2698468} \{stateful_app-count-store-name-changelog-0:
position=2655839, end=5743384, totalRestored=2655839}
state-updater [***-StateUpdater-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=5415824, end=5716195,
totalRestored=5415824} \{stateful_app-count-store-name-changelog-0:
position=5412953, end=5743384, totalRestored=5412953}
state-updater [***-StateUpdater-1] Finished restoring changelog
stateful_app-count-store-name-changelog-1 to store count-store-name with a
total number of 5716195 records
state-updater [***-StateUpdater-1] Restoration in progress for 1 partitions.
\{stateful_app-count-store-name-changelog-1: position=1000, end=5716195,
totalRestored=1000}
state-updater [***-StateUpdater-1] Finished restoring changelog
stateful_app-count-store-name-changelog-0 to store count-store-name with a
total number of 5743384 records
...
[Consumer clientId=***-StateUpdater-1-restore-consumer, groupId=null] Assigned
to partition(s): stateful_app-count-store-name-changelog-1,
stateful_app-count-store-name-changelog-0
...
[Consumer clientId=***-StateUpdater-1-restore-consumer, groupId=null] Resetting
offset for partition stateful_app-count-store-name-changelog-0 to position
FetchPosition{offset=0, offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[localhost:19092 (id: 1 rack:
null)], epoch=2}}.
was:
The logs for Kafka Streams local state restoration incorrectly refer to the
{{StreamThread}} instead of the {{StateUpdater}} thread, which is responsible
for decoupling the restoration process. The restore consumer group also
references {{StreamThread}} instead of {{{}StateUpdater{}}}, which should be
corrected for clarity.
*Current logs:*
...
stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=2698468, end=5716195,
totalRestored=2698468} \{stateful_app-count-store-name-changelog-0:
position=2655839, end=5743384, totalRestored=2655839}
stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=5415824, end=5716195,
totalRestored=5415824} \{stateful_app-count-store-name-changelog-0:
position=5412953, end=5743384, totalRestored=5412953}
stream-thread [***-StreamThread-1] Finished restoring changelog
stateful_app-count-store-name-changelog-1 to store count-store-name with a
total number of 5716195 records
stream-thread [***-StreamThread-1] Restoration in progress for 1 partitions.
\{stateful_app-count-store-name-changelog-1: position=1000, end=5716195,
totalRestored=1000}
stream-thread [***-StreamThread-1] Finished restoring changelog
stateful_app-count-store-name-changelog-0 to store count-store-name with a
total number of 5743384 records
...
[Consumer clientId=***-StreamThread-1-restore-consumer, groupId=null] Assigned
to partition(s): stateful_app-count-store-name-changelog-1,
stateful_app-count-store-name-changelog-0
...
[Consumer clientId=***-StreamThread-1-restore-consumer, groupId=null] Resetting
offset for partition stateful_app-count-store-name-changelog-0 to position
FetchPosition\{offset=0, offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[localhost:19092 (id: 1 rack:
null)], epoch=2}}.
*Expected logs:*
...
stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=2698468, end=5716195,
totalRestored=2698468} \{stateful_app-count-store-name-changelog-0:
position=2655839, end=5743384, totalRestored=2655839}
stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
\{stateful_app-count-store-name-changelog-1: position=5415824, end=5716195,
totalRestored=5415824} \{stateful_app-count-store-name-changelog-0:
position=5412953, end=5743384, totalRestored=5412953}
stream-thread [***-StreamThread-1] Finished restoring changelog
stateful_app-count-store-name-changelog-1 to store count-store-name with a
total number of 5716195 records
stream-thread [***-StreamThread-1] Restoration in progress for 1 partitions.
\{stateful_app-count-store-name-changelog-1: position=1000, end=5716195,
totalRestored=1000}
stream-thread [***-StreamThread-1] Finished restoring changelog
stateful_app-count-store-name-changelog-0 to store count-store-name with a
total number of 5743384 records
...
[Consumer clientId=***-StateUpdater-1-restore-consumer, groupId=null] Assigned
to partition(s): stateful_app-count-store-name-changelog-1,
stateful_app-count-store-name-changelog-0
...
[Consumer clientId=***-StateUpdater-1-restore-consumer, groupId=null] Resetting
offset for partition stateful_app-count-store-name-changelog-0 to position
FetchPosition\{offset=0, offsetEpoch=Optional.empty,
currentLeader=LeaderAndEpoch{leader=Optional[localhost:19092 (id: 1 rack:
null)], epoch=2}}.
> Improve the State Updater logs when restoring state
> ---------------------------------------------------
>
> Key: KAFKA-17744
> URL: https://issues.apache.org/jira/browse/KAFKA-17744
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Sebastien Viale
> Assignee: Sebastien Viale
> Priority: Minor
>
> The logs for Kafka Streams local state restoration incorrectly refer to the
> {{StreamThread}} instead of the {{StateUpdater}} thread, which is responsible
> for decoupling the restoration process. The restore consumer group also
> references {{StreamThread}} instead of {{{}StateUpdater{}}}, which should be
> corrected for clarity.
> *Current logs:*
> ...
> stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
> \{stateful_app-count-store-name-changelog-1: position=2698468, end=5716195,
> totalRestored=2698468} \{stateful_app-count-store-name-changelog-0:
> position=2655839, end=5743384, totalRestored=2655839}
> stream-thread [***-StreamThread-1] Restoration in progress for 2 partitions.
> \{stateful_app-count-store-name-changelog-1: position=5415824, end=5716195,
> totalRestored=5415824} \{stateful_app-count-store-name-changelog-0:
> position=5412953, end=5743384, totalRestored=5412953}
> stream-thread [***-StreamThread-1] Finished restoring changelog
> stateful_app-count-store-name-changelog-1 to store count-store-name with a
> total number of 5716195 records
> stream-thread [***-StreamThread-1] Restoration in progress for 1 partitions.
> \{stateful_app-count-store-name-changelog-1: position=1000, end=5716195,
> totalRestored=1000}
> stream-thread [***-StreamThread-1] Finished restoring changelog
> stateful_app-count-store-name-changelog-0 to store count-store-name with a
> total number of 5743384 records
> ...
> [Consumer clientId=***-StreamThread-1-restore-consumer, groupId=null]
> Assigned to partition(s): stateful_app-count-store-name-changelog-1,
> stateful_app-count-store-name-changelog-0
> ...
> [Consumer clientId=***-StreamThread-1-restore-consumer, groupId=null]
> Resetting offset for partition stateful_app-count-store-name-changelog-0 to
> position FetchPosition{offset=0, offsetEpoch=Optional.empty,
> currentLeader=LeaderAndEpoch{leader=Optional[localhost:19092 (id: 1 rack:
> null)], epoch=2}}.
>
> *Expected logs:*
> ...
> ...
> state-updater [***-StateUpdater-1] Restoration in progress for 2 partitions.
> \{stateful_app-count-store-name-changelog-1: position=2698468, end=5716195,
> totalRestored=2698468} \{stateful_app-count-store-name-changelog-0:
> position=2655839, end=5743384, totalRestored=2655839}
> state-updater [***-StateUpdater-1] Restoration in progress for 2 partitions.
> \{stateful_app-count-store-name-changelog-1: position=5415824, end=5716195,
> totalRestored=5415824} \{stateful_app-count-store-name-changelog-0:
> position=5412953, end=5743384, totalRestored=5412953}
> state-updater [***-StateUpdater-1] Finished restoring changelog
> stateful_app-count-store-name-changelog-1 to store count-store-name with a
> total number of 5716195 records
> state-updater [***-StateUpdater-1] Restoration in progress for 1 partitions.
> \{stateful_app-count-store-name-changelog-1: position=1000, end=5716195,
> totalRestored=1000}
> state-updater [***-StateUpdater-1] Finished restoring changelog
> stateful_app-count-store-name-changelog-0 to store count-store-name with a
> total number of 5743384 records
> ...
> [Consumer clientId=***-StateUpdater-1-restore-consumer, groupId=null]
> Assigned to partition(s): stateful_app-count-store-name-changelog-1,
> stateful_app-count-store-name-changelog-0
> ...
> [Consumer clientId=***-StateUpdater-1-restore-consumer, groupId=null]
> Resetting offset for partition stateful_app-count-store-name-changelog-0 to
> position FetchPosition{offset=0, offsetEpoch=Optional.empty,
> currentLeader=LeaderAndEpoch{leader=Optional[localhost:19092 (id: 1 rack:
> null)], epoch=2}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
