yashmayya commented on code in PR #16885:
URL: https://github.com/apache/pinot/pull/16885#discussion_r2380050291
##########
pinot-core/src/main/java/org/apache/pinot/core/operator/transform/function/CastTransformFunction.java:
##########
@@ -191,7 +194,19 @@ private long[] transformToTimestampValuesSV(ValueBlock
valueBlock) {
int length = valueBlock.getNumDocs();
initLongValuesSV(length);
String[] stringValues =
_transformFunction.transformToStringValuesSV(valueBlock);
- ArrayCopyUtils.copyToTimestamp(stringValues, _longValuesSV, length);
+ RoaringBitmap nullBitmap = _transformFunction.getNullBitmap(valueBlock);
Review Comment:
Even if query time null handling is disabled, these default string null
values will cause the cast transform to fail. IMO it's the right behavior to
convert it to epoch value in that case too. We also do the same thing when null
handling is enabled, but there the null bitmap will actually end up being used
and the final query result for the `null` rows would be `null`.
Both these cases are illustrated in the new tests I added to
`NullQueriesFluentTest` btw.
Edit: Discussed offline and our goal is to not introduce any performance
overhead when null handling is disabled (default case). So we should never
iterate over the null bitmap if null handling is disabled. So users have a
choice to either enable query time null handling for such cast queries on null
values, or else use `IS NOT NULL` filter predicate to remove these null values,
which works even without query time null handling (see
https://docs.pinot.apache.org/developers/advanced/null-value-support#basic-null-support)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
