andygrove opened a new pull request, #4162: URL: https://github.com/apache/datafusion-comet/pull/4162
## Which issue does this PR close? Closes #. ## Rationale for this change The current regular expressions section of the compatibility guide is two sentences long. It tells the user that Comet uses the Rust regexp crate and falls back to Spark by default, but does not explain why the two engines differ or what to look out for when deciding whether to opt in via `spark.comet.expression.regexp.allowIncompatible=true`. Users hitting fallbacks have no high-level reference for the kinds of pattern features that are not portable between Java and Rust. ## What changes are included in this PR? Expands `docs/source/user-guide/latest/compatibility/regex.md` with: - A restatement of why Comet falls back by default and which expressions are affected. - A short explanation of the architectural difference between Java's PCRE-style backtracking engine and the Rust regex crate's RE2-style finite-automaton engine. - A list of features Java supports that the Rust engine does not (backreferences, lookaround, atomic groups, possessive quantifiers, conditionals and recursion). - A list of features that exist on both sides but behave differently (Unicode-aware character classes, line terminators, case folding, POSIX class syntax, octal/Unicode escape syntax, empty-match split semantics). - Guidance on when enabling the incompatible flag is generally safe. ## How are these changes tested? Documentation only. No code or behavior changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
