Ankitkj1999 opened a new pull request, #4356: URL: https://github.com/apache/iceberg-python/pull/4356
Closes #4355 # Rationale for this change When a partition field is removed (`update_spec().remove_field(...)`) and its source column is later dropped (`update_schema().delete_column(...)`), historical partition specs and manifests still reference the dropped `source_id`. Previously: - `TableMetadata.specs_struct()` only queried `self.schema().find_type(field.source_id)`, causing `ValueError` in `entries()`, `partitions()`, `files()`, `data_files()`, `delete_files()`, `all_files()`, `all_data_files()`, and `all_delete_files()`. - `InspectTable._generate_manifests_table()` evaluated `spec.partition_type(self.tbl.schema())`, which resolved dropped source columns as `UnknownType()` and rendered historical `partition_summaries` bounds as `'null'`. This change updates `TableMetadata.specs_struct()` to fall back through historical schemas in `reversed(self.schemas)` (and `UnknownType()` only if absent from all schemas), and updates `InspectTable._generate_manifests_table()` to reuse `specs_struct([spec.spec_id])`. ## Are these changes tested? Yes: - Added unit tests in `tests/table/test_metadata.py` (`test_specs_struct_resolves_dropped_partition_source_from_historical_schemas`, `test_specs_struct_falls_back_to_unknown_when_source_missing_from_all_schemas`). - Added integration/inspection tests in `tests/table/test_inspect.py` (`test_inspect_after_dropping_old_partition_column`, `test_inspect_after_promoting_then_dropping_partition_column`). ## Are there any user-facing changes? Bugfix: `table.inspect` tables remain queryable and preserve manifest partition summary bounds after dropping an old partition source column. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
