kishoreg opened a new issue #5416:
URL: https://github.com/apache/incubator-pinot/issues/5416
We have multiple types of Functions in Pinot
- AggregationFunction
- Used during query execution
- Stateful
- aggregate method can be invoked multiple times with blockValMap
- follows map/combine/reduce paradigm
- TransformFunction
- used during query execution
- stateless
- transform method is invoked with projectionBlock as input
- outputs another block (length of output block is same as input block)
- IngestionTransformFunction
- Used during ingestion (real-time and batch)
- stateless
- takes Generic Row and returns one value
- ScalarFunction (this is basically any java function)
- These are pure java functions that take a bunch of arguments and return a
value
Most of the utility functions should be represented as ScalarFunctions
IngestTransformFunction, TransformFunction can be syntactic sugars on top of
these scalar functions.
We are seeing the same function being needed in transform, ingestion, query
parsing (e.g. date time functions) and we have to write the same version
multiple times. Should we write generic wrappers around Scalar Function for
ingestion, transform?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
