dhruvarya-db opened a new pull request, #18783: URL: https://github.com/apache/iceberg/pull/18783
This PR builds on #18782, and its first commit contains the changes from #18782. To review only the changes in this PR, use https://github.com/dhruvarya-db/iceberg/compare/3c43da7bce1816a74a4a4a7828160dedbb40903d...rest-v4-table-location-upstream. ## Why In format version 4, the `location` field in table metadata is optional. Paths in the metadata can also be relative to the table location. A client must know the table location to find these files and to write new files. The format spec says that the catalog must supply the location when the metadata does not include it. This change is outlined in the REST spec PR for format version 4, #17642. It adds a `table-location` field for this purpose to load, commit, and unregister responses. ## What changes The Java REST client reads `table-location` from load, commit, and unregister responses. If the field is present, the client uses it as the table location, instead of the `location` field in the metadata. The client does not send this location back to the server in later commits, so the catalog keeps control of the location. The client keeps the response as it was received, so the same response gives the same JSON again. The client rejects a `table-location` that is not an absolute path with a URI scheme. It also rejects a version 4 response that has no location at all. Without this check, the error occurs later in a write, and the cause is hard to find. The server code in `CatalogHandlers` sends `table-location` only for version 4 tables. Responses for versions 1 to 3 do not change. One result can be unexpected. A create transaction can start from a staged response that gets its location from `table-location`. The create commit then sends that location to the server. This is necessary, because the create commit is the step that saves the location of the new table. ## Testing **New coverage** - `TestLoadTableResponseParser` and `TestUnregisterTableResponseParser` have new tests. They check JSON with and without `table-location`, and that `table-location` overrides `location`. - The same tests check that the client records no metadata change, and that it rejects a relative location or no location. - `TestCatalogHandlersTableLocation` is a new suite. It checks that the server sends `table-location` only for version 4 tables. - `TestRESTCatalogFormatV4` is a new end-to-end suite. It loads a version 4 table without `location` over HTTP and appends data to it. - This test checks that the client uses the location from the catalog, and that the commit does not send the location back. It fails if the server stops sending `table-location`. **Coverage from the current suites** The current parser tests and REST suites use tables with format versions 1 to 3. They pass with no change, which shows that responses for these versions are the same as before. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: [unknown - human to fill in] - Prompt Summary: Carry a catalog-supplied table location on REST table responses and apply it to the client's table metadata for format version 4 tables. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
