andygrove commented on issue #5685: URL: https://github.com/apache/datafusion-comet/issues/5685#issuecomment-5546996766
#5667 is now rebased on merged `main`. I measured what it does to the cases above, so this issue can be scoped to what is left rather than guessed at. `CometExplodeBenchmark`, `main` at `719cba110` against the rebased branch, both built and run back to back on one machine: Apple M3 Ultra, JDK 17, `local[1]`, Spark 4.1 / Scala 2.13, release native build (Best ms): | Case | Spark | Comet before | Comet after | Before | After | | ---- | ----- | ------------ | ----------- | ------ | ----- | | `explode array<struct<a bigint, b string>>[10]` | 57 | 65 | 54 | 0.9X | **1.1X** | | `explode array<struct>[4]` at depth 1 | 63 | 114 | 97 | 0.6X | 0.7X | | `explode array<struct>[4]` at depth 8 | 77 | 117 | 100 | 0.7X | 0.8X | | `explode array<struct>[4]` at depth 8, then its inner array | 83 | 155 | 115 | 0.5X | 0.7X | The Spark arm moves by at most 3ms between the two runs across the whole benchmark, so it holds as a control and the movement is Comet's. The flat `array<struct>` case, the one in the title, crosses over: 0.9X to 1.1X. The three nested cases improve by 15% to 26% and stay behind Spark at 0.7X to 0.8X. So #5667 closes the first row and narrows the rest without closing them, which is what this issue anticipated. It should stay open for the nested gap. The numbers here are on a different machine from the ones in the issue body, which were an M3 Max, so read the before/after columns against each other rather than against the table above. One thing outside this issue that the same run turned up: `explode array<bigint>[100]` was also behind Spark on `main` (148ms against 127, 0.9X) and is 112ms / 1.1X after. That was not visible in the original M3 Max run, where the case was not regressed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
