Gabriel39 opened a new pull request, #66504: URL: https://github.com/apache/doris/pull/66504
### What problem does this PR solve? File Scanner V2 can produce hundreds of one-row selection ranges after compound raw filtering. Dictionary decoding then repeatedly enters skip, decode, and validation for every range. The roughly 5% fragmented Decimal predicate shape in TPC-DS Q28 misses the existing dense-selection threshold and spends significant CPU on dictionary-index transitions. ### What is changed? - Preserve the existing dense-selection policy. - Decode dictionary IDs sequentially once when the estimated range-transition cost exceeds one full pass and the index scratch buffer fits in L2 cache. - Keep clustered and very sparse selections on range decode. - Add a 5% decoder benchmark axis and a regression test for the Q28-shaped selection. ### Performance Release build, same-host CPU-pinned A-B-B-A comparison, 9 repetitions per run. Values are average paired median CPU time; lower is better. | Scenario | Before (ns) | After (ns) | Change | |---|---:|---:|---:| | Dictionary INT32, 5% alternating | 191,996 | 25,664 | -86.63% | | Dictionary INT32, 10% alternating | 375,928 | 34,112 | -90.93% | | Dictionary INT64, 5% alternating | 234,943 | 31,844 | -86.45% | | Dictionary BYTE_ARRAY, 5% alternating | 230,975 | 30,649 | -86.73% | | Dictionary INT32, 1% alternating (control) | 68,290 | 44,670 | -34.59% | | Dictionary INT32, 5% clustered (control) | 26,223 | 18,698 | -28.70% | | Plain INT32, 5% alternating (control) | 11,344 | 10,781 | -4.96% | No regressions were observed in the control matrix. Existing paths remain unchanged unless fragmented-selection transition cost crosses the new cache-aware threshold. ### Validation - 132 targeted Parquet native-decoder and benchmark-scenario tests passed. - Release microbenchmark completed with the matrix above. - Release BE build completed successfully. This PR is stacked on #66485; the final commit contains this change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
