123digits opened a new issue, #3340: URL: https://github.com/apache/iceberg-rust/issues/3340
### Is your feature request related to a problem or challenge? **Iceberg Java already does this.** Its REST client loads tables "freshness-aware": it keeps a table cache, sends `If-None-Match` with the cached `ETag`, and reuses the cached table when the server answers `304 Not Modified`. The cache is configured by two catalog properties in [`RESTCatalogProperties`](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/rest/RESTCatalogProperties.java): - `rest-table-cache.max-entries` (default 100) - `rest-table-cache.expire-after-write-ms` (default 5 minutes) The same is requested for iceberg-go (apache/iceberg-go#2124) and pyiceberg (apache/iceberg-python#4075). **What the spec defines** (`open-api/rest-catalog-open-api.yaml`): - `loadTable` (`GET /v1/{prefix}/namespaces/{namespace}/tables/{table}`) accepts an optional `If-None-Match` header: "allows the server to return 304 (Not Modified) if the metadata is current. The content is the value of the ETag received in a CreateTableResponse, LoadTableResponse or CommitTableResponse." - `LoadTableResponse` carries an `etag` header. - `304`: "Not Modified - Based on the content of the 'If-None-Match' header the table metadata has not changed since." **What iceberg-rust does today** (0.10.1 and `main`): `RestCatalog::load_table` sends no `If-None-Match` and keeps no `ETag` from load, create or commit responses. **A related bug:** `load_table` matches `StatusCode::OK | StatusCode::NOT_MODIFIED` together and deserializes a `LoadTableResult` from either. A `304` has no body, so if a server (or a proxy) ever answers `304`, the load fails with a deserialization error instead of reusing a cached table. **Use case:** clients that load the same tables repeatedly, such as Kubernetes operators reconciling on an interval, long-running services and query engines, download and parse the full metadata JSON on every `load_table`, even when nothing changed. That JSON grows with snapshot history and can be large for busy tables. Servers already answer conditionally: Lakekeeper sends an `ETag` and replies `304` from v0.11.0. ### Describe the solution you'd like Matching Iceberg Java, so the same catalog properties work across clients: - An optional, bounded, per-catalog cache of identifier → (`ETag`, loaded metadata and metadata location), filled from the `etag` header of load, create and commit responses. - When an entry exists, `load_table` sends `If-None-Match` and, on `304`, returns the cached table instead of deserializing the empty body. - The cache is configured by Java's property names, `rest-table-cache.max-entries` and `rest-table-cache.expire-after-write-ms`. ### Willingness to contribute I cannot contribute to this feature at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
