[GitHub] [pinot] nizarhejazi commented on a diff in pull request #9173: Proper null handling in equality, inequality and membership operators for all SV column data types

[GitBox]

nizarhejazi commented on code in PR #9173:
URL: https://github.com/apache/pinot/pull/9173#discussion_r939438715


##########
pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/SVScanDocIdIterator.java:
##########
@@ -204,6 +288,34 @@ public int matchValues(int limit, int[] docIds) {
     }
   }
 
+  private class IntMatcherAndNullHandler implements ValueMatcher {
+
+    private final ImmutableRoaringBitmap _nullBitmap;
+    private final int[] _buffer = new int[OPTIMAL_ITERATOR_BATCH_SIZE];
+
+    public IntMatcherAndNullHandler(ImmutableRoaringBitmap nullBitmap) {
+      _nullBitmap = nullBitmap;
+    }
+
+    @Override
+    public boolean doesValueMatch(int docId) {
+      if (_nullBitmap.contains(docId)) {
+        // Any comparison (equality, inequality, or membership) with null 
results in false (similar to Presto) even if

Review Comment:
   For nulls to get considered, one of 4 operators can be used:
   
   1. `IS NULL`
   2. `IS NOT NULL`
   3. `IS DISTINCT FROM`           => We need to implement it in Apache Pinot
   4. `IS NOT DISTINCT FROM`  => We need to implement it in Apache Pinot
   
   Please check unit tests as they demonstrate the behaviour if IS NULL or IS 
NOT NULL are used.



##########
pinot-core/src/main/java/org/apache/pinot/core/operator/dociditerators/SVScanDocIdIterator.java:
##########
@@ -204,6 +288,34 @@ public int matchValues(int limit, int[] docIds) {
     }
   }
 
+  private class IntMatcherAndNullHandler implements ValueMatcher {
+
+    private final ImmutableRoaringBitmap _nullBitmap;
+    private final int[] _buffer = new int[OPTIMAL_ITERATOR_BATCH_SIZE];
+
+    public IntMatcherAndNullHandler(ImmutableRoaringBitmap nullBitmap) {
+      _nullBitmap = nullBitmap;
+    }
+
+    @Override
+    public boolean doesValueMatch(int docId) {
+      if (_nullBitmap.contains(docId)) {
+        // Any comparison (equality, inequality, or membership) with null 
results in false (similar to Presto) even if

Review Comment:
   For nulls to get considered, one of 4 operators can be used:
   
   1. `IS NULL`
   2. `IS NOT NULL`
   3. `IS DISTINCT FROM`           => We need to implement it in Apache Pinot
   4. `IS NOT DISTINCT FROM`  => We need to implement it in Apache Pinot
   
   Please check unit tests as they demonstrate the behaviour if `IS NULL` or 
`IS NOT NULL` are used.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org