Alaksiej Ščarbaty created NIFI-15759:
----------------------------------------
Summary: Handle ResourceNotFoundException gracefully in
ConsumeKinesis for deleted shards
Key: NIFI-15759
URL: https://issues.apache.org/jira/browse/NIFI-15759
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 2.8.0
Reporter: Alaksiej Ščarbaty
Assignee: Alaksiej Ščarbaty
When a Kinesis stream is resharded or deleted, accessing shards that no longer
exist results in ResourceNotFoundException from the AWS SDK. Currently, neither
the Enhanced Fan-Out (EFO) nor the polling consumer handles this exception,
resulting in:
• Repeated ERROR-level log messages until the 60-second shard cache TTL
expires and the shard is dropped from the active set;
• In EFO mode, the ShardConsumer continuously resubscribes to the deleted
shard every 5 seconds;
• Unnecessary noise in logs for what is expected operational behavior during
resharding.
The processor should stop consuming data from a Shard whenever
ResourceNotFoundException is encounted.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
