Github user manuzhang commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/67#discussion_r72598874
--- Diff:
external/kafka/src/main/scala/org/apache/gearpump/streaming/kafka/lib/source/AbstractKafkaSource.scala
---
@@ -74,11 +74,10 @@ abstract class AbstractKafkaSource(
private lazy val kafkaClient: KafkaClient =
kafkaClientFactory.getKafkaClient(config)
private lazy val fetchThread: FetchThread =
fetchThreadFactory.getFetchThread(config, kafkaClient)
private lazy val messageDecoder = config.getConfiguredInstance(
- KafkaConfig.MESSAGE_DECODER_CLASS_CONFIG, classOf[MessageDecoder])
- private lazy val timestampFilter = config.getConfiguredInstance(
--- End diff --
previously filter is used to filter out old messages (e.g. timestamp <
startTime) and carried out in KafkaSource implicitly. Now I think it should be
defined explicitly in the following operations by users like
`withAllowedLateness` in Beam API although that is not available in Gearpump
API yet.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
