Kumar Mallikarjuna created FLINK-39382:
------------------------------------------
Summary: [Docs][Kafka] Clarify properties.auto.offset.reset
support in SQL connector
Key: FLINK-39382
URL: https://issues.apache.org/jira/browse/FLINK-39382
Project: Flink
Issue Type: Improvement
Components: Connectors / Kafka
Reporter: Kumar Mallikarjuna
The current documentation for the Flink SQL Kafka Connector contains a
misleading statement regarding the {{auto.offset.reset}} property that
contradicts the actual behavior of the connector since Flink 1.15
(specifically, the resolution of FLINK-24697).
*Current Documentation Statement:* On the [Kafka SQL Connector
page|https://nightlies.apache.org/flink/flink-docs-stable/docs/connectors/table/kafka/],
the docs state:
{quote}_"But there are some configurations that do not support to set, because
Flink will override them, e.g. 'auto.offset.reset'."_
{quote}
While it is true that Flink overrides the underlying Kafka Consumer client's
{{auto.offset.reset}} property to {{none}} (to protect its own state
management), the statement implies that setting
{{properties.auto.offset.reset}} in the Flink DDL is useless or entirely
unsupported.
Because of the fix introduced in FLINK-24697, this is no longer accurate. When
a user configures {{{}scan.startup.mode = 'group-offsets'{}}}, the Flink SQL
connector explicitly parses {{properties.auto.offset.reset}} from the DDL to
act as the fallback initialization strategy when no committed offsets are found
for the consumer group.
https://github.com/apache/flink-connector-kafka/blob/bd2fd2b2c8a48b7d2a05387fc4fda61e878c902f/flink-connector-kafka/src/main/java/org/apache/flink/streaming/connectors/kafka/table/KafkaDynamicSource.java#L453-L456
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
