priyen opened a new issue, #9402:
URL: https://github.com/apache/pinot/issues/9402
We have this query, ```
SELECT
"customer_id"
FROM
"table name"
WHERE
"_viewing_merchant" = 'some merchant id'
AND ((
"customer_id" < 'some customer id'
AND "last_payment" = 1661489443000.0
)
OR "last_payment" < 1661489443000.0
)```
and this is the explain on it
```
BROKER_REDUCE(sort:[last_payment DESC, customer_id DESC],limit:21) 0
-1
COMBINE_SELECT_ORDERBY 1 0
SELECT_ORDERBY(selectList:last_payment, customer_id) 2 1
TRANSFORM_PASSTHROUGH(customer_id, last_payment) 3 2
PROJECT(last_payment, customer_id) 4 3
FILTER_AND 5 4
FILTER_INVERTED_INDEX(indexLookUp:inverted_index,operator:EQ,predicate:_viewing_merchant
= 'some merchant id') 6 5
FILTER_OR 7 5
FILTER_AND 8 7
FILTER_FULL_SCAN(operator:RANGE,predicate:customer_id < 'xxxxxxx') 9
8
FILTER_FULL_SCAN(operator:EQ,predicate:last_payment = '1661489443000')
10 8
FILTER_FULL_SCAN(operator:RANGE,predicate:last_payment < '1661489443000')
11 7
```
with the last_payment = clause, the result has
`numEntriesScannedInFilter=42846997`, and if we remove it, then
`numEntriesScannedInFilter=554` and it finishes in ms instead of hundreds of
ms.
Seems like we are doing full scan on all data instead of just the one
returned by the inverted index.
The performance diff is from hundreds of ms to just ms if this is resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
