amrishlal opened a new pull request #6927: URL: https://github.com/apache/incubator-pinot/pull/6927
## Description This PR does type casting of numerical types only for GREATER_THAN, GREATER_THAN_OR_EQUAL, LESS_THAN, and LESS_THAN_OR_EQUAL operators. A previous PR (see #6811) did numerical type casting for EQ and NOT_EQ operators. Future PRs will extend numerical type conversion to include other operators (IN, NOT_INT, and HAVING clause). Queries with predicates that compare a column expression of one numerical type (INT for example) with a literal of another numerical type (DOUBLE for example) will fail to evaluate on the server side: ``` SELECT count(*) FROM testTable WHERE intColumn < 1.0 SELECT count(*) FROM testTable WHERE intColumn <= 1.0 SELECT count(*) FROM testTable WHERE intColumn > 15.1 SELECT count(*) FROM testTable WHERE intColumn >= 15.1 ``` This happens because we currently don't support upcasting and downcasting of numerical predicates with different data types. To solve this problem, we rewrite the predicate on the Broker side to an equivalent predicate whose literal type is same as the column expression type before the query is send to the server for evaluation. Also, there are some cases where we can tell whether the filter will always evaluate to TRUE or FALSE. If FilterOptimizer determines that the filter will always evaluate to FALSE, the Broker will return an empty result back to the user without server-side query evaluation. Similarly, if FilterOptimizer determines that the filter will always evaluate to TRUE, the Broker will remove the WHERE clause before sending the query to the server. **Example 1: Simple conversion** The query `SELECT count(*) FROM testTable WHERE intColumn < 1.0` will be rewritten to `SELECT count(*) FROM testTable WHERE intColumn < 1` before server-side query evaluation. **Example 2: Predicate precomputed to false** The query `SELECT count(*) FROM testTable WHERE intColumn > 3000000000.5` will always evaluate to FALSE since no integer column can have decimal value greater than 3000000000.5. In this case the FilterOptimizer will rewrite the query to `SELECT count(*) FROM testTable WHERE FALSE` and Broker will send an empty response back to the user without server-side query evaluation. **Example 3: Predicate precomputed to true** The query `SELECT count(*) FROM testTable WHERE intColumn <= 3000000000.5` will always evaluate to TRUE since an integer column will always have values than 3000000000.5. In this case the FilterOptimizer will rewrite the query to `SELECT count(*) FROM testTable WHERE TRUE` and Broker will send an empty response back to the user without server-side query evaluation. **Example 4: Predicate operator modified** The query `SELECT count(*) FROM testTable WHERE intColumn < 300.5` will be rewritten to `SELECT count(*) FROM testTable WHERE intColumn <= 300`. Note that the interger counterpart (300) of a decimal value (300.5) is always less than the original value. Hence operator '<' is replaced with operator '<='. Similarly, the query `SELECT count(*) FROM testTable WHERE intColumn < -300.5` will be rewritten to `SELECT count(*) FROM testTable WHERE intColumn <= -300` The query `SELECT count(*) FROM testTable WHERE intColumn > -300.5` will be rewritten to `SELECT count(*) FROM testTable WHERE intColumn > -300`, and so on... ## Upgrade Notes Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion) * [ ] Yes (Please label as **<code>backward-incompat</code>**, and complete the section below on Release Notes) Does this PR fix a zero-downtime upgrade introduced earlier? * [ ] Yes (Please label this as **<code>backward-incompat</code>**, and complete the section below on Release Notes) Does this PR otherwise need attention when creating release notes? Things to consider: - New configuration options - Deprecation of configurations - Signature changes to public methods/interfaces - New plugins added or old plugins removed * [ ] Yes (Please label this PR as **<code>release-notes</code>** and complete the section on Release Notes) ## Release Notes <!-- If you have tagged this as either backward-incompat or release-notes, you MUST add text here that you would like to see appear in release notes of the next release. --> <!-- If you have a series of commits adding or enabling a feature, then add this section only in final commit that marks the feature completed. Refer to earlier release notes to see examples of text. --> ## Documentation <!-- If you have introduced a new feature or configuration, please add it to the documentation as well. See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
