featzhang created FLINK-39366:
---------------------------------
Summary: [connector-http] Add retry support with exponential
backoff for HTTP Sink
Key: FLINK-39366
URL: https://issues.apache.org/jira/browse/FLINK-39366
Project: Flink
Issue Type: New Feature
Components: Connectors / HTTP
Reporter: featzhang
h2. Summary
Add retry support for HTTP Sink connector. When HTTP requests fail due to
network errors (IOException), the sink should automatically retry with
exponential backoff strategy, improving fault tolerance and reliability in
unstable network environments.
h2. Motivation
The current HTTP Sink implementation does not retry failed requests. Once a
network error occurs, the failed requests are counted in error metrics but not
re-submitted, leading to silent data loss. A configurable retry mechanism with
exponential backoff is needed to handle transient failures gracefully.
h2.
Proposed Changes
Add SINKHTTPRETRY_TIMES* constant to HttpConnectorConfigConstants
Add RETRY_TIMES* ConfigOption to HttpDynamicSinkConnectorOptions (default: 3)
Register RETRY_TIMES* as an optional option in HttpDynamicTableSinkFactory
* Pass retry configuration via Properties in HttpDynamicSink
* Implement exponential backoff retry logic in HttpSinkWriter:
** Retry only on IOException (network-level errors)
** Exponential backoff with initial delay of 1 second (1s → 2s → 4s → ...)
** Set retry.times=0 to disable retries
* Update unit tests to cover retry behavior
* Update documentation (English and Chinese) with new option description
h2. New Configuration Option
Option Key Default Description
| sink.http.retry.times | 3 | Number of retry attempts on IOException. Set to 0
to disable. |
h2. Acceptance Criteria
HttpSinkWriter retries failed requests up to the configured number of times on
IOException.
Retry interval follows exponential backoff (initial delay 1s, doubles each
attempt).
Setting retry.times=0 disables retry behavior.
All existing tests pass; new unit tests cover retry scenarios.
Documentation updated in both English and Chinese.
h2. Related
* PR: https://github.com/apache/flink-connector-http/pull/33
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
