RaidenE1 opened a new pull request, #20251:
URL: https://github.com/apache/kafka/pull/20251
**Problem**
Kafka Streams immediately throws `MissingSourceTopicException` when source
topics are missing during rebalance, but in distributed systems there can be
timing issues where topics exist but metadata hasn't fully propagated. This
causes
failures in scenarios where a brief retry period would resolve the issue.
**Solution**
Add `missing.source.topic.timeout.ms config` (default: 30s) to allow
configurable grace period before failing on missing source topics.
**Behavior**:
- `-1`: Immediate failure (existing behavior, default in tests)
- `≥0`: Retry for specified duration, then fail
**Implementation**
- StreamsPartitionAssignor: During grace period throws
`TaskAssignmentException` (retry), after timeout throws
`MissingSourceTopicException` (fail)
- Configuration: Added to `StreamsConfig` with proper validation
- Tests: Added unit tests for timeout behavior, all existing tests pass
**Files Changed**
- `StreamsConfig.java` - New config
- `StreamsPartitionAssignor.java` - Timeout logic
- `AssignorConfiguration.java` - Config exposure
- `StreamsRebalanceListener.java` - Simplified constructor
- `StreamThread.java` - Constructor updates
- Test files - New timeout tests + config defaults
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
