mcvsubbu commented on issue #7004: URL: https://github.com/apache/incubator-pinot/issues/7004#issuecomment-884336631
There are many other methods to get the Kafka (or a stream) lag, so I don't consider that as big plus. In fact, we are experimenting with something that we will put out a PR on soon. I am clearly not a +1000, I am curious what the others in PPMC (or even community) think about adding a feature like this. If implemented, it has to be done very carefully to make sure no extra garbage is generated even when the feature is turned off (needless to say, it has to be a configurable feature). I am not sure I fully understand the UDF/real column that you propose, but it sounds promising, and should be looked at. Are you proposing to use a special decoder? We can extend the stream interface to include such UDFs, so that the underlying implementation may decide to populate such columns using such a decoder. Another thing to consider is whether this can be done sparsely (i.e. sample every so many records). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
