richardstartin commented on pull request #7820: URL: https://github.com/apache/pinot/pull/7820#issuecomment-982151840
> Conceptually, we are moving from a Column to Column + path but modelling the path as an evaluator. But the query layer is deciding the evaluator instead of the storage layer. Specifically on this, not really, the model has the potential to be far more generic than that. It’s more going from column to column + parameters + arbitrary sequence of storage layer API calls. This is good, because within the context of an evaluator, the most efficient API calls can be made. Case in point: in DefaultJsonPathEvaluator it’s possible to avoid converting byte[] to String. It would also be possible to use hypothetical new zero copy (but unsafe) methods to read bytes on the basis that we don’t need to create a vector of JSON records, but a vector of JSON evaluation results. Individual evaluator implementations can be arbitrarily efficient (and complex) in a way that prevents the query layer from ever seeing any of this complexity, keeping the query layer clean just as it avoids trying to model all of the complex access patterns we might like in the storage layer with a single interface. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
