kishoreg commented on pull request #7820:
URL: https://github.com/apache/pinot/pull/7820#issuecomment-982088481
I like the concept of the pushdown evaluator. Few thoughts/concerns
- Some methods added to projectionBlock seem to be specific to
projectionBlock and is deviating from the block concept of chaining any blocks
together i.e. the caller of the block does not understand the implementation of
the underlying block. We have violated this in few places and we should fix
that. It might be a good idea to enhance the Block interface to allow a
pushdown evaluator.
- Conceptually, we are moving from a Column to Column + path but modelling
the path as an evaluator. But the query layer is deciding the evaluator instead
of the storage layer.
- The evaluator seems to be set at the JVM level but in practice, it's
possible for each table to have a different evaluator and in fact, it's
possible that each segment can have a different evaluator.
At a high level, the segment is the smallest unit of query processing in
Pinot and most of the things can be decided independently at that scope -
segmentDirectory, readers, writers, Blocks, indexes, encoding etc.
Having said that, I don't have a very good solution yet but I think we can
come up with something after a couple of iterations. One thought is enhancing
the concept (String Column) to a Column Object which has SingleColumn,
NestedColumn( column + path) as implementations.
Overall, I think the Storage (Reader) should decide on the evaluator instead
of the query layer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
