andygrove opened a new pull request, #4907:
URL: https://github.com/apache/datafusion-comet/pull/4907
> This PR was created by an LLM as a draft PR. I will mark it as ready for
review after human review.
## Which issue does this PR close?
N/A — autonomous exploratory PR.
## Rationale for this change
Replaced the full serde_json::Value materialization of each row with a
streaming DeserializeSeed path walk that skips non-matching subtrees via
IgnoredAny and allocates only the matched subtree, eliminating per-key and
per-value heap allocations for the whole document.
## What changes are included in this PR?
Replaced the full serde_json::Value materialization of each row with a
streaming DeserializeSeed path walk that skips non-matching subtrees via
IgnoredAny and allocates only the matched subtree, eliminating per-key and
per-value heap allocations for the whole document.
## How are these changes tested?
Correctness: unit tests + seeded differential fuzz (bit-identical Arrow
output vs `main`).
Benchmark (criterion):
- nested_field: 75.342% faster (base 7961939ns -> cand 1963265ns)
- top_level_field: 76.842% faster (base 7968457ns -> cand 1845346ns)
- array_index: 75.091% faster (base 7562066ns -> cand 1883639ns)
Full criterion output:
```text
get_json_object/top_level_field
time: [1.8425 ms 1.8451 ms 1.8481 ms]
change: [−76.936% −76.842% −76.753%] (p = 0.00 <
0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
3 (3.00%) low mild
3 (3.00%) high mild
4 (4.00%) high severe
get_json_object/nested_field
time: [1.9589 ms 1.9639 ms 1.9682 ms]
change: [−75.504% −75.342% −75.196%] (p = 0.00 <
0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
5 (5.00%) low mild
1 (1.00%) high mild
get_json_object/array_index
time: [1.8813 ms 1.8854 ms 1.8907 ms]
change: [−75.236% −75.091% −74.962%] (p = 0.00 <
0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
