nmshr opened a new pull request, #4087:
URL: https://github.com/apache/iceberg-python/pull/4087
# Rationale for this change
The `remove_statistics()` method on the statistics update builder is
dropping updates which are queued before it.
This PR supersedes #3557
```python
table.update_statistics() \
.set_statistics(statistics_file) \
.remove_statistics(other_snapshot_id) \
.commit()
```
When `remove_statistics()` is called, the update queued by
`set_statistics()` gets dropped, so at commit, only the remove is preserved.
The side effects of this could be that the query engines plan with stale
statistics.
This happens as `=` is used in `remove_statistics()` method. `=` replaces
the builder's list of pending updates instead of appending to it.
`set_statistics()` is using `+=`.
The fix is to use `+=` in `remove_statistics()` method, like in
`set_statistics()`.
## Are these changes tested?
Yes, `test_update_statistics_set_then_remove_keeps_both_updates`
(tests/table/test_init.py) has been added as a new unit test.
Consider a table that has snapshots s2 and s1, each with its statistics
files.
The test sets a new statistics file for s2, then removes s1's statistics.
After the commit, s2 points to the new file and s1 has no statistics. Both
changes are applied.
On the main branch, this test fails, the `set_statistics()` for s2 gets
dropped, s2 keeps pointing to its old file.
With the fix, make test, 4,254 passed, and make lint passing.
## Are there any user-facing changes?
No API changes. set-then-remove now applies both updates.
## AI disclosure
AI was used to write the code, tests and for local verification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
