[GitHub] [pinot] nizarhejazi commented on a diff in pull request #8927: Proper null handling in SELECT, ORDER BY, DISTINCT, and GROUP BY

[GitBox]

nizarhejazi commented on code in PR #8927:
URL: https://github.com/apache/pinot/pull/8927#discussion_r917456694


##########
pinot-core/src/main/java/org/apache/pinot/core/operator/blocks/IntermediateResultsBlock.java:
##########
@@ -390,28 +452,57 @@ private DataTable getAggregationResultDataTable()
       columnNames[i] = aggregationFunction.getColumnName();
       columnDataTypes[i] = 
aggregationFunction.getIntermediateResultColumnType();
     }
+    RoaringBitmap[] nullBitmaps = null;
+    Object[] colDefaultNullValues = null;
+    if (_isNullHandlingEnabled) {
+      colDefaultNullValues = new Object[numAggregationFunctions];
+      nullBitmaps = new RoaringBitmap[numAggregationFunctions];
+      for (int i = 0; i < numAggregationFunctions; i++) {
+        if (columnDataTypes[i] != ColumnDataType.OBJECT) {
+          // Store a dummy value that is both a valid numeric, and a valid hex 
encoded value.
+          String specialVal = "30";
+          colDefaultNullValues[i] = 
columnDataTypes[i].toDataType().convert(specialVal);
+        }
+        nullBitmaps[i] = new RoaringBitmap();
+      }
+    }
 
     // Build the data table.
     DataTableBuilder dataTableBuilder =
         DataTableFactory.getDataTableBuilder(new DataSchema(columnNames, 
columnDataTypes));
     dataTableBuilder.startRow();
     for (int i = 0; i < numAggregationFunctions; i++) {
+      Object value = _aggregationResult.get(i);
+      // OBJECT (e.g. DistinctTable) calls toBytes() (e.g. 
DistinctTable.toBytes()) which takes care of replacing nulls
+      // with default values, and building presence vector and serializing 
both.
+      if (_isNullHandlingEnabled && columnDataTypes[i] != 
ColumnDataType.OBJECT) {

Review Comment:
   We need to override null value before calling switch statement (casting to 
Number and calling longValue/doubleValue throws an exception).
   Btw, this method operators on aggregation results which has a very low 
dimension compared to the raw data dimension.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org