mbutrovich opened a new pull request, #23573: URL: https://github.com/apache/datafusion/pull/23573
## Which issue does this PR close? - Backport of #22950 to `branch-54` (for 54.1.0, tracked in #22547). - Fixes panic #22935 (reporter bisected to a regression between 53.1.0 and 54.0.0; introduced by #20047). ## Rationale for this change Anonymous file reads can read the same path with different explicit schemas in one session. The shared file-statistics cache did not validate that cached statistics matched the read schema, so a narrower cached entry could be reused for a later wider-schema read and panic during statistics projection (`index out of bounds` in projection). No API change, so it fits the backport criteria. ## What changes are included in this PR? This is an **adapted** backport. The core fix (`table.rs`, `options.rs`, `context/mod.rs`) is applied; two conflicts were resolved for `branch-54`'s API shape: - `ListingTable::statistics_cache()` returns `Option<&Arc<dyn FileStatisticsCache>>` (on `branch-54`, `FileStatisticsCache` is a trait, not a concrete type as on `main`). - Kept `branch-54`'s import paths for `TableScopedPath` / `FileStatisticsCache`. - The regression test `anonymous_parquet_stats_cache_with_explicit_wider_schema` computes statistics via `plan.partition_statistics(None)` instead of `statistics_with_args`, which does not exist on `branch-54`. ## Are these changes tested? Yes. The regression test passes with the fix and, without it, reproduces the exact reported panic (`index out of bounds: the len is 2 but the index is 2`), confirming it exercises the regression. ## Are there any user-facing changes? No API change. Fixes a panic on anonymous multi-schema reads of the same path. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
