sungwy opened a new issue, #3978:
URL: https://github.com/apache/iceberg-python/issues/3978
`pyiceberg/manifest.py` holds a module-level cache:
```python
_manifest_cache = _ManifestCache()
```
Its docstring describes it accurately: *"Process-wide ManifestFile cache
keyed by manifest_path."* The key is the path string alone — no catalog, table,
session, or tenant forms part of it.
```python
def get_or_cache(self, manifest_file: ManifestFile) -> ManifestFile:
...
manifest_path = manifest_file.manifest_path
if manifest_path in self._cache:
return self._cache[manifest_path]
self._cache[manifest_path] = manifest_file
return manifest_file
```
On a hit it returns the cached object and discards the one just read. Since
`read_manifest_list` constructs `ManifestFile` objects from the entries of a
*manifest list* rather than by reading each manifest file, the cached object
reflects whatever the first manifest list to name that path declared —
including its partition summaries, counts and sequence numbers.
Two tables in the same process that reference the same `manifest_path`
therefore share one `ManifestFile`, and the first read wins. Because those
fields drive scan pruning, a stale or mismatched entry changes which files a
scan considers.
---
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
