npawar opened a new issue, #8488: URL: https://github.com/apache/pinot/issues/8488
The filter config needs a function that evaluates to boolean https://docs.pinot.apache.org/developers/advanced/ingestion-level-transformations#filtering So far, we were able to use Groovy for expressing such fucntions. Starting release 0.10.0, Groovy is disabled by default, and also due to security concerns might not be the best choice to use even if explicitly enabled. This leaves us with only being able to use regular Scalar Functions in `filterFunction`. However, we don't have sufficient scalar functions which evaluate to boolean (only ones I could find are `startsWith` and `endsWith` in StringFunctions. Goal: Add all commonly used comparison functions (like equals, no equals, greater than, less than etc). We should be able to express such functions completely without Groovy `"filterFunction": "Groovy({timestamp < 1589007600000}, timestamp)"`, `"filterFunction": "Groovy({(campaign == \"X\" || campaign == \"Y\") && prices.sum() < 100}, prices, campaign)"` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
