ErikBPF commented on issue #5870: URL: https://github.com/apache/datafusion-comet/issues/5870#issuecomment-5769764409
`verify-benchmark-results-tpcds` runs three full query passes on every full run — sort-merge, broadcast-hash, shuffled-hash (`pr_build_linux.yml:813-828`). Measured 7m19s + 4m36s + 7m03s = **18m58s**; the PR-tier TPC-DS job was 23m49s in #6003. All three compare against the same golden output, so the extra passes add join-strategy coverage, not output coverage. Suggestion: default pass only on the PR tier, all three on the merge queue/nightly — ~12 min/PR run × ~56 runs ≈ **670 runner-min/72 h (~1 %)**. Caveat: a PR could then pass and red the queue on a non-default strategy, which is the failure this issue exists to reduce, so it belongs with the tiering decision. Happy to open the PR if wanted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
