Nemesis123925 opened a new issue, #11939: URL: https://github.com/apache/pinot/issues/11939
## Setup: Java version: openjdk 11.0.20.1 Maven version: Apache Maven 3.8.1 ## Fixed Test: org.apache.pinot.queries.ExplainPlanQueriesTest#testSelect org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregate org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterGroupBy org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterGroupByVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndex org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexGroupBy org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexGroupByHaving org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexGroupByHavingVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexGroupByOrderBy org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexGroupByOrderByVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexGroupByVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterIndexVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterOnTextIndexColumn org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateUsingFilterOnTextIndexColumnVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectAggregateVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnUsingFilterOnJsonIndexColumnVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnUsingFilterOnRangeIndexColumn org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnUsingFilterOnRangeIndexColumnVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsUsingFilter org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsUsingFilterOnInvertedIndexColumn org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsUsingFilterOnInvertedIndexColumnVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsUsingFilterVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsVariationsOfAndOperators org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsVariationsOfAndOperatorsVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsVariationsOfOrOperators org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectColumnsVariationsOfOrOperatorsVerbose org.apache.pinot.queries.ExplainPlanQueriesTest#testSelectVerbose ## Test Failure: The test's `PROJECT` field's order shows nondeterminism when applying the `NonDex` tool. When asserting the created query's `PROJECT` field, the column names's order could be random, which `PROJECT(invertedIndexCol1, noIndexCol1) ` could randomly be `PROJECT(noIndexCol1, invertedIndexCol1)`. Also, when the create query consisted of nested queries, the two nested queries's order could also be random. ## Root Cause: The root cause is in [pinot-core/src/main/java/org/apache/pinot/core/operator/ProjectionOperator.java's `toExplainString()` function](https://github.com/Nemesis123925/pinot/blob/24af80d3bccdb4c2b86cae392ea022b824db1c1a/pinot-core/src/main/java/org/apache/pinot/core/operator/ProjectionOperator.java#L86), which it iterate through the `keySet` of `_dataSourceMap`, which is created as a `HashMap`. This `_dataSourceMap` is created in [pinot-core/src/main/java/org/apache/pinot/core/plan/ProjectPlanNode.java](https://github.com/Nemesis123925/pinot/blob/ad0d2b173a67ac19bc39120179f48192c80520e4/pinot-core/src/main/java/org/apache/pinot/core/plan/ProjectPlanNode.java#L74) and its order is determined by [pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionUtils.java](https://github.com/Nemesis123925/pinot/blob/ad0d2b173a67ac19bc39120179f48192c80520e4/pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/AggregationFunctionUtils.jav a#L88), which are whether `HashMap` or `HashSet` ## Fix: The Fix here is by replacing the `HashSet` by `LinkedHashSet` and `HashMap` by `LinkedHashSet` in the above position. This complete eliminates the nondeterminism inside. This will also disturb the hardcoded `result` in the tests, so I also change the order in the `result` array in the tests, and make the tests pass. However, I am here to ask is it acceptable to change the main code? If not, I can replace the fix by only changing the test code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
