kishoreg commented on a change in pull request #5765:
URL: https://github.com/apache/incubator-pinot/pull/5765#discussion_r462002553
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/ProjectionBlock.java
##########
@@ -25,22 +25,22 @@
import org.apache.pinot.core.common.BlockMetadata;
import org.apache.pinot.core.common.BlockValSet;
import org.apache.pinot.core.common.DataBlockCache;
-import org.apache.pinot.core.common.DataSourceMetadata;
+import org.apache.pinot.core.common.DataSource;
import org.apache.pinot.core.operator.docvalsets.ProjectionBlockValSet;
-import org.apache.pinot.spi.data.FieldSpec;
/**
* ProjectionBlock holds a column name to Block Map.
* It provides DocIdSetBlock for a given column.
*/
public class ProjectionBlock implements Block {
- private final Map<String, DataSourceMetadata> _dataSourceMetadataMap;
+ private final Map<String, DataSource> _dataSourceMap;
Review comment:
probably not relevant to this PR but we should try to move towards using
Block instead of DataSource
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/docvalsets/ProjectionBlockValSet.java
##########
@@ -32,32 +35,33 @@
public class ProjectionBlockValSet implements BlockValSet {
private final DataBlockCache _dataBlockCache;
private final String _column;
- private final DataType _dataType;
- private final boolean _singleValue;
+ private final DataSource _dataSource;
/**
* Constructor for the class.
- * The dataBlockCache argument is initialized in {@link ProjectionOperator},
- * so that it can be reused across multiple calls to {@link
ProjectionOperator#nextBlock()}.
- *
- * @param dataBlockCache data block cache
- * @param column Projection column.
+ * The dataBlockCache is initialized in {@link ProjectionOperator} so that
it can be reused across multiple calls to
+ * {@link ProjectionOperator#nextBlock()}.
*/
- public ProjectionBlockValSet(DataBlockCache dataBlockCache, String column,
DataType dataType, boolean singleValue) {
+ public ProjectionBlockValSet(DataBlockCache dataBlockCache, String column,
DataSource dataSource) {
_dataBlockCache = dataBlockCache;
_column = column;
- _dataType = dataType;
- _singleValue = singleValue;
+ _dataSource = dataSource;
}
@Override
public DataType getValueType() {
- return _dataType;
+ return _dataSource.getDataSourceMetadata().getDataType();
}
@Override
public boolean isSingleValue() {
- return _singleValue;
+ return _dataSource.getDataSourceMetadata().isSingleValue();
+ }
+
+ @Nullable
+ @Override
+ public Dictionary getDictionary() {
Review comment:
how come we never needed this? For group by, we already have the
optimization to work on dictionary ids right?
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/query/aggregation/function/DistinctCountAggregationFunction.java
##########
@@ -212,11 +259,57 @@ public IntOpenHashSet
extractGroupByResult(GroupByResultHolder groupByResultHold
IntOpenHashSet valueSet = groupByResultHolder.getResult(groupKey);
if (valueSet == null) {
return new IntOpenHashSet();
+ }
+
+ if (_dictionary != null) {
+ // For dictionary-encoded expression, convert dictionary ids to values
+ return convertToValueSet(valueSet);
} else {
return valueSet;
}
}
+ private IntOpenHashSet convertToValueSet(IntOpenHashSet dictIdSet) {
Review comment:
java doc for this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
