andygrove commented on issue #5685: URL: https://github.com/apache/datafusion-comet/issues/5685#issuecomment-5547191237
I decomposed the depth-1 nested case to find out where the remaining time actually goes, and the answer reframes this issue: the generator is a small part of it. The dominant term is the nested Parquet scan, which is #4859. All numbers below are the same machine and build as my previous comment (Apple M3 Ultra, JDK 17, `local[1]`, Spark 4.1 / Scala 2.13, release native, 100K rows of the benchmark's `nested` view), on this branch with #5667 applied. Best ms. ### Stripping the case down | Query | Spark | Comet | | | ----- | ----- | ----- | --- | | `count(k)`, flat column only | 17 | 10 | 1.6X | | `count(events)`, scan the whole struct, no generator at all | 60 | 92 | 0.7X | | plus `explode`, no carried columns | 61 | 111 | 0.6X | | plus carried `k, region`, which is this issue's `depth 1` case | 63 | 113 | 0.6X | Of Comet's 113ms, 92ms is already spent before any generator exists. `ExplodeExec` adds about 19ms and the carried-column gather about 2ms. Spark's generator adds close to nothing, because whole-stage codegen fuses it into the scan loop, but it is a small absolute number under either engine. Further work on the operator is chasing a fifth of the gap. ### It is not general scan overhead, it is nested pruning Comet wins on the flat column, so this is specific to nested reads. And this pair is the tell: | Query | Spark | Comet | | | ----- | ----- | ----- | --- | | `count(events)`, 5 leaves | 60 | 92 | 0.7X | | `count(events.platform)`, 1 leaf | 22 | 93 | 0.2X | Spark drops to a third of its time when only one leaf is needed. Comet does not move at all. Both executed plans print the identical pruned `ReadSchema: struct<events:array<struct<platform:string>>>`, so the pruning reaches the plan and then has no effect on what Comet reads. That is #4859. ### What #5262 does to it Same queries built and run on `apache/df55`: | Query | Spark (main / df55) | Comet on main | Comet on df55 | | ----- | ------------------- | ------------- | ------------- | | `count(events)` whole struct | 60 / 65 | 92 (0.7X) | **59 (1.1X)** | | `count(events.platform)` one leaf | 22 / 25 | 93 (0.2X) | **13 (1.9X)** | | `count(events.entries)` inner array | 61 / 63 | 93 (0.7X) | **57 (1.1X)** | | `explode` then count one leaf | 23 / 34 | 95 (0.2X) | **21 (1.6X)** | Pruning works there, and the whole-struct read also improves where there is nothing to prune, so DF55 helps the nested reader beyond leaf selection alone. One caveat for anyone repeating this: `df55` has no `explode.rs`, it predates `CometExplodeExec` by 114 commits, so a generator query on that branch falls back to Spark's `GenerateExec`. I measured the full nested case at 0.5X there and discarded it as a different code path. Only the scan-only rows above are valid across the two branches. ### Suggested order 1. Land #5262. Nothing available in `ExplodeExec` competes with 92ms to 59ms on the whole struct, or 93ms to 13ms on a pruned leaf. 2. Re-measure this issue afterwards before doing more operator work. Composing what is measured above, DF55's 59ms scan plus main's roughly 19ms of explode against Spark's roughly 63ms, projects these cases to around 0.8X: better than today's 0.6X to 0.7X, and possibly still short of parity. Neither branch has both halves today, so nobody has seen the combined number yet. 3. Only then is the residual worth attacking, and it would be the carried-column gather and the aggregate over the four-times-expanded batch rather than `unnest_list_array`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
