sunithabeeram commented on a change in pull request #4270: skip the inverted
index generation
URL: https://github.com/apache/incubator-pinot/pull/4270#discussion_r289875293
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java
##########
@@ -158,8 +158,12 @@ public void init(SegmentGeneratorConfig
segmentCreationSpec, SegmentIndexCreatio
}
// Initialize inverted index creator
+ // Do not created inverted index if sorted, clean up
invertedIndexColumns
if (invertedIndexColumns.contains(columnName)) {
- if (segmentCreationSpec.isOnHeap()) {
+ if (indexCreationInfo.isSorted()){
+ invertedIndexColumns.remove(columnName);
+ //config.getInvertedIndexCreationColumns().remove(columnName); Do
we clean up config?
Review comment:
The way the execution code is currently structured, for us to use
SortedInvertedIndexBasedFilterOperator, we still need the hint for
inverted-index hint - so I don't think you can remove it from config. Please
trace the code and ensure that that's not broken. Also make sure there are
tests covering the execution for sorted columns.
BaseSingleValueQueriesTest.java is a comprehensive test suite that should
cover this, but it looks like we are not covering the case there where a column
is both sorted and has invertedIndex. You can extend that test to cover this
case and ensure things are working as expected. You will most likely also need
a test to assert that the bitmap based index isn't generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
