github-actions[bot] commented on code in PR #67051:
URL: https://github.com/apache/doris/pull/67051#discussion_r3843031213
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/source/LancePredicateConverter.java:
##########
@@ -257,6 +268,60 @@ private Optional<Expression> convertIsNull(IsNullPredicate
predicate) {
return Optional.of(comparisonFunction(function,
fieldReference(field)));
}
+ private Optional<Expression> convertLike(LikePredicate predicate) {
+ if (predicate.getOp() != LikePredicate.Operator.LIKE) {
+ return Optional.empty();
+ }
+ return convertStringPredicate("like:str_str", predicate.getChild(0),
predicate.getChild(1), true);
+ }
+
+ private Optional<Expression> convertStringFunction(FunctionCallExpr
function) {
+ if (function.getFnName() == null || function.getChildren().size() !=
2) {
+ return Optional.empty();
+ }
+ String functionName =
function.getFnName().getFunction().toLowerCase(Locale.ROOT);
Review Comment:
Preserve the resolved function identity before dispatching by name.
FunctionRegistry intentionally allows a UDF to shadow a built-in via
prefer_udf_over_builtin, while a qualified call selects the UDF directly, and
Nereids translates Java/Python UDFs to this same FunctionCallExpr with a
non-BUILTIN catalog function. A two-argument UDF named starts_with, ends_with,
or like can therefore reach this switch, be replaced by Lance's built-in, and
be removed from the Doris residual in LanceScanNode; an empty residual can also
enable limit pushdown. Please require a non-null resolved Doris BUILTIN before
the switch (leaving unknown/UDF identities residual) and cover an analyzed
same-named UDF.
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/source/LancePredicateConverter.java:
##########
@@ -257,6 +268,60 @@ private Optional<Expression> convertIsNull(IsNullPredicate
predicate) {
return Optional.of(comparisonFunction(function,
fieldReference(field)));
}
+ private Optional<Expression> convertLike(LikePredicate predicate) {
+ if (predicate.getOp() != LikePredicate.Operator.LIKE) {
+ return Optional.empty();
+ }
+ return convertStringPredicate("like:str_str", predicate.getChild(0),
predicate.getChild(1), true);
+ }
+
+ private Optional<Expression> convertStringFunction(FunctionCallExpr
function) {
+ if (function.getFnName() == null || function.getChildren().size() !=
2) {
+ return Optional.empty();
+ }
+ String functionName =
function.getFnName().getFunction().toLowerCase(Locale.ROOT);
+ switch (functionName) {
+ case "like":
+ return convertStringPredicate(
+ "like:str_str", function.getChild(0),
function.getChild(1), true);
+ case "starts_with":
+ return convertStringPredicate(
+ "starts_with:str_str", function.getChild(0),
function.getChild(1), false);
+ case "ends_with":
+ return convertStringPredicate(
+ "ends_with:str_str", function.getChild(0),
function.getChild(1), false);
+ default:
+ return Optional.empty();
+ }
+ }
+
+ private Optional<Expression> convertStringPredicate(
+ String function, Expr input, Expr pattern, boolean
rejectEscapedPattern) {
+ SlotRef slot = directSlot(input);
+ LiteralExpr literal = directLiteral(pattern);
+ ResolvedField field = slot == null ? null : findField(slot);
+ if (field == null || !isStringType(field.field.getType()) || !(literal
instanceof StringLiteral)) {
+ return Optional.empty();
+ }
+ String patternValue = literal.getStringValue();
+ // Doris uses backslash as LIKE's default escape character, while the
Substrait function
+ // has no escape argument. Keep escaped LIKE patterns in Doris rather
than changing meaning.
+ if (rejectEscapedPattern && patternValue.indexOf('\\') >= 0) {
Review Comment:
Keep embedded-NUL LIKE patterns residual. In the default SQL mode Doris
decodes backslash-zero before this guard; a wildcard pattern then reaches
hs_compile through re_pattern.c_str(), so the regex is truncated at NUL (for
example, a<NUL>_ becomes ^a and matches abc). The exact lance-c 0.1.6 /
DataFusion 53.1.0 / Arrow 58.3.0 consumer preserves the full NUL-bearing
pattern, so pushdown filters abc out and the removed Doris residual cannot
restore it. Please reject NUL here and add an analyzed end-to-end
residual/result case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
