siddharthteotia opened a new pull request #6288: URL: https://github.com/apache/incubator-pinot/pull/6288
Currently we build sorted index only if the column is dictionary encoded. However, when we write isSorted in on-disk segment metadata, we write on the basis of pre-index stats collector. So, for a sorted column without dictionary, segment metadata will indicate column as sorted `properties.setProperty(getKeyFor(column, IS_SORTED), String.valueOf(columnIndexCreationInfo.isSorted()));` During query processing, when we create filter operator, we check the data source metadata to see if the column is sorted and create sorted index based filter operator. However, using this operator for any sorted raw column will lead to the following error stack since we end up using a raw value based predicate evaluator for a dictionary based filter operator. The solution is to do the additional check on data source to see if the column is dictionary encoded or not `java.lang.UnsupportedOperationException 181762 at org.apache.pinot.core.operator.filter.predicate.BaseRawValueBasedPredicateEvaluator.getMatchingDictIds(BaseRawValueBasedPredicateEvaluator.java:40) 181763 at org.apache.pinot.core.operator.filter.SortedIndexBasedFilterOperator.getNextBlock(SortedIndexBasedFilterOperator.java:68) 181764 at org.apache.pinot.core.operator.filter.SortedIndexBasedFilterOperator.getNextBlock(SortedIndexBasedFilterOperator.java:35) 181765 at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49) 181766 at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:62) 181767 at org.apache.pinot.core.operator.DocIdSetOperator.getNextBlock(DocIdSetOperator.java:35) 181768 at org.apache.pinot.core.operator.BaseOperator.nextBlock(BaseOperator.java:49) 181769 at org.apache.pinot.core.operator.ProjectionOperator.getNextBlock(ProjectionOperator.java:57) 181770 at org.apache.pinot.core.operator.ProjectionOperator.getNextBlock(ProjectionOperator.java:30)` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
