[
https://issues.apache.org/jira/browse/KAFKA-8388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843233#comment-16843233
]
Richard Yu edited comment on KAFKA-8388 at 5/18/19 8:01 PM:
------------------------------------------------------------
To your first question, the answer I think is yes.
The TTL part I don't think that directly applies here. Time-to-live basically
means that entries exist for a guaranteed time interval right? The intention of
this issue is to store entries that was inserted within a specific time
interval which the user defines in a separate KTable.
The idea here is allow the user to persist and access entries within a specific
interval at their discretion. The input/result KTable would be constantly
updated, so eventually older entries that existed in the KTable would be wiped
out. If the user chooses to preserve these older entries, they could be allowed
to do so by inserting them in the new KTable.
In effect, the user could control to some extent which key-values are
maintained in-memory. Basically, time-to-live policies means that the cache
controls how long an entry persists, while in this case, the user gets to
choose which elements are saved.
was (Author: yohan123):
To your first question, the answer I think is yes.
The TTL part I don't think that directly applies here. Time-to-live basically
means that entries exist for a guaranteed time interval right? The intention of
this issue is to store entries that was inserted within a specific time
interval in a separate KTable for reference by the user.
The idea here is allow the user to persist and access entries within a specific
interval at their discretion. The input/result KTable would be constantly
updated, so eventually older entries that existed in the KTable would be wiped
out. If the user chooses to preserve these older entries, they could be allowed
to do so by inserting them in the new KTable.
In effect, the user could control to some extent which key-values are
maintained in-memory. Basically, time-to-live policies means that the cache
controls how long an entry persists, while in this case, the user gets to
choose which elements are saved.
> Add methods to query for entries in KTable using timestamp
> ----------------------------------------------------------
>
> Key: KAFKA-8388
> URL: https://issues.apache.org/jira/browse/KAFKA-8388
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Richard Yu
> Priority: Major
> Labels: needs-kip
>
> Currently, in Kafka Streams, KTable API does not allow users to query for
> entries with a specific timestamp. The goal of this ticket is to allow a
> couple of things:
> * Retrieve entries in KTable that were inserted in the last {{K}}
> milliseconds where {{K}} could be determined by the user
> * Find key-value pairs which were put into the KTable in a specified time
> interval.
> KTable already has methods like {{filter}} and {{filterNot}}, but the
> information available to the user in these methods does not include the
> publish timestamp.
> The best way to do this is probably to add new methods which creates a new
> KTable based solely on timestamp.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
