ankitsultana commented on code in PR #15498:
URL: https://github.com/apache/pinot/pull/15498#discussion_r2049189342
##########
pinot-common/src/main/java/org/apache/pinot/sql/parsers/ParserUtils.java:
##########
@@ -39,6 +39,49 @@ public static void validateFunction(String canonicalName,
List<Expression> opera
}
}
+ /**
+ * Sanitize the sql string for parsing by normalizing whitespace which can
+ * cause performance issues with regex based parsing.
+ * This method replaces multiple consecutive whitespace characters with a
single space.
+ *
+ * @param sql The raw SQL string to sanitize. May be null.
+ * @return A sanitized SQL string with normalized whitespace and no trailing
spaces,
+ * or {@code null} if the input was {@code null}.
+ */
+ public static String sanitizeSqlForParsing(String sql) {
+
+ // 1. Remove excessive whitespace
+
+ int length = sql.length();
+ StringBuilder builder = new StringBuilder(length);
+ boolean inWhitespaceBlock = false;
+
+ for (int charIndex = 0; charIndex < length; charIndex++) {
Review Comment:
@jitendrakr88 : we can do this to extract options:
* Remove *all* spaces from the query.
* Then extract out the trailing options. Options don't rely on spaces
anyways and are completely token based.
This should address all issues I believe and keep implementation simple too.
For extraction you could first find the last occurrence of option keyword and
then use the regex. This will also guard against someone trying to inject
`option option option..`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
