yjhjstz opened a new pull request, #1966: URL: https://github.com/apache/cloudberry/pull/1966
When a memo group's cached statistics lack columns needed by a later derivation request (e.g. a ROLLUP expanded into a CTE whose consumers request different column sets), the missing columns are derived separately and merged via CStatistics::AppendStats(). That merge copied histograms and widths only, so the colid -> attno mapping (added for extended statistics) diverged from the histogram map: a column could have a histogram but no attno entry. ApplyCorrelatedStatsToScaleFactorFilterCalculation() then dereferenced the result of the attno lookup without a null check and crashed the coordinator with SIGSEGV whenever the table had any extended statistics object with stored data and a filter referenced one of the appended columns (e.g. TPC-DS Q67 after CREATE STATISTICS (ndistinct) on item/date_dim). Reproduced on main with a cassert build: colid 11 had a histogram (5 entries) but the attno map only held 4 entries, attnum was NULL at CExtendedStatsProcessor.cpp:280. This is the same class of bug as the CTE remapping fix in 124a048e8c6 (gpdb#16212), which repaired CopyStatsWithRemap() but not AppendStats(). Fix both layers: * CStatistics::AppendStats() now also merges the colid -> attno mapping (copy-on-write, since the mapping is shared by refcount across stats objects, see ScaleStats()) and adopts the input's extended-stats info when none is set, restoring the invariant that every column with a histogram has an attno entry. * CExtendedStatsProcessor now null-checks the attno and histogram lookups and skips clauses it cannot translate, falling back to the independence assumption, matching what ApplyCorrelatedStatsToNDistinctCalculation() already does. Also bail out gracefully if no ndistinct item matches the attribute set. * CExtendedStatsProcessor also skips system columns (attno <= 0): extended statistics never cover them, and feeding a negative attno into CBitSet::ExchangeSet() would convert it to a huge unsigned bit index. The backend rejects these clauses too (dependency_is_compatible_clause()). Add a regression test with the minimized reproduction: a ROLLUP over a join where the dimension table has ndistinct extended statistics and the filter column is appended to the scan group's stats after the initial derivation. Verified the gporca test passes with both optimizer=on and optimizer=off answer files. Ported from warehouse-pg/warehouse-pg#251. Fixes #1949 ### What does this PR do? <!-- Brief overview of the changes, including any major features or fixes --> ### Type of Change - [ ] Bug fix (non-breaking change) - [ ] New feature (non-breaking change) - [ ] Breaking change (fix or feature with breaking changes) - [ ] Documentation update ### Breaking Changes <!-- Remove if not applicable. If yes, explain impact and migration path --> ### Test Plan <!-- How did you test these changes? --> - [ ] Unit tests added/updated - [ ] Integration tests added/updated - [ ] Passed `make installcheck` - [ ] Passed `make -C src/test installcheck-cbdb-parallel` ### Impact <!-- Remove sections that don't apply --> **Performance:** <!-- Any performance implications? --> **User-facing changes:** <!-- Any changes visible to users? --> **Dependencies:** <!-- New dependencies or version changes? --> ### Checklist - [ ] Followed [contribution guide](https://cloudberry.apache.org/contribute/code) - [ ] Added/updated documentation - [ ] Reviewed code for security implications - [ ] This PR contains AI-assisted code generation - [ ] Requested review from [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers) ### Additional Context <!-- Any other information that would help reviewers? Remove if none --> ### CI Skip Instructions <!-- To skip CI builds, add the appropriate CI skip identifier to your PR title. The identifier must: - Be in square brackets [] - Include the word "ci" and either "skip" or "no" - Only use for documentation-only changes or when absolutely necessary --> --- <!-- Join our community: - Mailing list: [[email protected]](https://lists.apache.org/[email protected]) (subscribe: [email protected]) - Discussions: https://github.com/apache/cloudberry/discussions --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
