npawar opened a new issue #5509: URL: https://github.com/apache/incubator-pinot/issues/5509
Introduce derived columns - columns which are derived from other columns in the Pinot data record. This is not the same as simply having transform functions. Transform functions are applied to source columns (x) to get pinot columns (y) i.e. `y = f(x)` Derived columns are derived from pinot columns i.e. `y = f(y)`. By default, the derived columns will be materialized during ingestion. We also want an option, by which we don't materialize them, instead execute the transform function during query time (An example of where this would be useful - populate a column based on current time) Derived columns will also be useful in adding columns to existing data, if the column value can be derived from existing data. For instance, if we have column `foo`, and then add `bar` to the schema, if `bar = f(foo)`, we can backfill all of the data, without having to regenerate segment. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
