davidradl commented on code in PR #27589:
URL: https://github.com/apache/flink/pull/27589#discussion_r2794547148
##########
flink-core/src/main/java/org/apache/flink/configuration/PipelineOptions.java:
##########
@@ -328,4 +329,25 @@ public enum VertexDescriptionMode {
+ "while still using watermark alignment,
set this parameter to true. "
+ "The default value is false. Note: This
parameter may be "
+ "removed in future releases.");
+
+ @Experimental
+ public static final ConfigOption<Integer> WATERMARK_ALIGNMENT_BUFFER_SIZE =
+ key("pipeline.watermark-alignment.buffer-size")
+ .intType()
+ .defaultValue(3)
Review Comment:
I am curious to what value the user would know to set this to. I am thinking
the situation is something like:
we are processing events and watermarks, we see water marks 1 , 2 and 3,
then come across events that are now considered late because we have the
current watermark as 3, now if 1 were in the ring buffer and we check against
that then that event would not be considered late. Would there be a downside
to set this to 100 or a large number?
I am not sure how a user would ever be able to know how to set this. I
wonder if we has a list or queue of watermarks that we kept until we had
finished out processing, then we would always have the watermarks we need. In
this case we do not need the user to supply the number of entries in a ring
buffer, instead to dynamically keep track of the watermarks we need. Is this
feasible - or am I missing something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
