mxm opened a new pull request, #13963:
URL: https://github.com/apache/iceberg/pull/13963
Adds error stream functionality to the Flink dynamic sink system, allowing
applications to handle processing errors gracefully instead of failing the
entire job.
Benefits:
- Fault tolerance: Applications can continue processing valid records while
isolating problematic ones
- Observability: Failed records and their exceptions are captured for
debugging and monitoring
- Flexibility: Users can implement custom error handling logic (retry, dead
letter queue, logging, etc.)
- Existing applications continue to work unchanged; error stream is opt-in
Usage:
```
DynamicIcebergSink.forInput(dataStream)
.generator(new MyGenerator())
.catalogLoader(catalogLoader)
.errorStreamConsumer(errorStream -> {
// Handle errors
errorStream.sinkTo(..)
})
.append();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
