[GitHub] [pinot] walterddr commented on a diff in pull request #8927: Proper null handling in SELECT, ORDER BY, DISTINCT, and GROUP BY

[GitBox]

walterddr commented on code in PR #8927:
URL: https://github.com/apache/pinot/pull/8927#discussion_r957435868


##########
pinot-core/src/main/java/org/apache/pinot/core/query/selection/SelectionOperatorUtils.java:
##########
@@ -227,16 +229,42 @@ public static void 
mergeWithOrdering(PriorityQueue<Object[]> mergedRows, Collect
    *
    * @param rows {@link Collection} of selection rows.
    * @param dataSchema data schema.
+   * @param nullHandlingEnabled whether null handling is enabled.
    * @return data table.
    * @throws Exception
    */
-  public static DataTable getDataTableFromRows(Collection<Object[]> rows, 
DataSchema dataSchema)
+  public static DataTable getDataTableFromRows(Collection<Object[]> rows, 
DataSchema dataSchema,
+      boolean nullHandlingEnabled)
       throws Exception {
     ColumnDataType[] storedColumnDataTypes = 
dataSchema.getStoredColumnDataTypes();
     int numColumns = storedColumnDataTypes.length;
 
-    DataTableBuilder dataTableBuilder = DataTableFactory.getDataTableBuilder(
-        dataSchema);
+    DataTableBuilder dataTableBuilder = 
DataTableFactory.getDataTableBuilder(dataSchema);
+    RoaringBitmap[] nullBitmaps = null;
+    if (nullHandlingEnabled) {
+      nullBitmaps = new RoaringBitmap[numColumns];
+      Object[] colDefaultNullValues = new Object[numColumns];
+      for (int colId = 0; colId < numColumns; colId++) {
+        if (storedColumnDataTypes[colId] != ColumnDataType.OBJECT && 
!storedColumnDataTypes[colId].isArray()) {
+          colDefaultNullValues[colId] =
+              
NullValueUtils.getDefaultNullValue(storedColumnDataTypes[colId].toDataType());
+        }
+        nullBitmaps[colId] = new RoaringBitmap();
+      }
+
+      int rowId = 0;
+      for (Object[] row : rows) {
+        for (int i = 0; i < numColumns; i++) {
+          Object columnValue = row[i];
+          if (columnValue == null) {
+            row[i] = colDefaultNullValues[i];
+            nullBitmaps[i].add(rowId);
+          }
+        }
+        rowId++;

Review Comment:
   can we merge this part with the main loop below? any specific reason we had 
to loop through the rows one more time?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org