AMBRA7592 opened a new pull request, #4212: URL: https://github.com/apache/iceberg-python/pull/4212
Closes #4185 # Rationale for this change Removing s2 from snapshot-log [s1, s2, s3] currently leaves [s1, s3]. A timestamp lookup during s2 then incorrectly returns s1, even after the table is reloaded. Update RemoveSnapshotsUpdate to discard accumulated history whenever it encounters an expired snapshot. This retains only the log suffix after the last expired occurrence, as required by the [Iceberg specification](https://iceberg.apache.org/spec/#table-metadata). Correcting the persisted metadata avoids changing snapshot_as_of_timestamp itself. This does not change which snapshots are retained or their explicit snapshot-ID lookup. It does not reconstruct history already damaged by earlier expirations. Related work: #4206 preserves parent snapshot IDs in the same handler; #4201 adds expired-file cleanup. This change addresses snapshot-log history only and does not include either change. ## Are these changes tested? - Ten metadata-history scenarios across v1/v2/v3 fixtures with validation enabled, including gaps, repeated IDs after rollback, empty expiration, and off-main snapshots. - Catalog-backed regressions for expiration by ID and by age across three local catalog fixtures. They check timestamp boundaries, serialized metadata, reloads, protected tags, and retained/current data. Only clocks are controlled; commits, file IO, and scans are real. - Python 3.12.13, macOS arm64, all extras synchronized from uv.lock: full unit suite 4,484 passed, 5 Windows-only skips, 129 non-unit cases deselected. Coverage: 90.62%, above the 85% threshold. - Full make lint, including mypy, passed. uv lock --check and git diff --check passed. - Related suites on Python 3.14.2: 292 passed, using a separate environment with a subset of the test dependencies. - Restoring the original removal handler in a separate process makes all nine new collected regression tests fail with assertions. They pass with this fix; each of the three metadata tests loops over ten scenarios. These are local results, not upstream CI. Linux/Windows, Python 3.11/3.13, external-service integration suites, the documentation build, and the Java RAT checker were not run. The metadata fixtures are not a full format-interoperability matrix. ## Are there any user-facing changes? Yes. After expiration, timestamp lookups without retained historical coverage return None under the existing Python helper contract, instead of silently returning an older snapshot. Earlier log entries before the expired gap are removed; surviving snapshots remain readable by ID. AI assistance: Codex was used for the implementation, regression tests, and local test execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
