andygrove opened a new pull request, #23547:
URL: https://github.com/apache/datafusion/pull/23547
> This PR was created by an LLM as a draft PR. I will mark it as ready for
review after human review.
## Which issue does this PR close?
N/A — autonomous exploratory PR.
## Rationale for this change
Pass a literal regexp pattern/flags to arrow's kernel as scalar Datums so
the regex is compiled once per batch, instead of expanding the literal to a
full array and forcing a per-row HashMap<String, Regex> cache lookup (plus a
per-row format! allocation when flags are present).
## What changes are included in this PR?
Pass a literal regexp pattern/flags to arrow's kernel as scalar Datums so
the regex is compiled once per batch, instead of expanding the literal to a
full array and forcing a per-row HashMap<String, Regex> cache lookup (plus a
per-row format! allocation when flags are present).
## Are these changes tested?
Correctness: unit tests + seeded differential fuzz (bit-identical Arrow
output vs `main`).
Benchmark (criterion):
- regexp_match_1000 literal pattern utf8view: 21.394% faster (base 249069ns
-> cand 195784ns)
- regexp_match_1000 pattern array: 2.996% faster (base 190020ns -> cand
184327ns)
- regexp_match_1000 literal pattern: 23.964% faster (base 261795ns -> cand
199059ns)
- regexp_match_1000 literal pattern and flags: 37.227% faster (base 280047ns
-> cand 175794ns)
Full criterion output:
```text
regexp_match_1000 literal pattern
time: [198.39 µs 198.66 µs 199.03 µs]
change: [−24.244% −23.964% −23.600%] (p = 0.00 <
0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe
regexp_match_1000 literal pattern and flags
time: [175.59 µs 175.68 µs 175.81 µs]
change: [−37.342% −37.227% −37.124%] (p = 0.00 <
0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
regexp_match_1000 literal pattern utf8view
time: [195.75 µs 195.85 µs 195.98 µs]
change: [−21.511% −21.394% −21.298%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
3 (3.00%) high mild
1 (1.00%) high severe
regexp_match_1000 pattern array
time: [184.11 µs 184.22 µs 184.37 µs]
change: [−3.1007% −2.9962% −2.8581%] (p = 0.00 <
0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) high mild
4 (4.00%) high severe
```
## Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!--
If there are any breaking changes to public APIs, please add the `api
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
