dkranchii opened a new pull request, #17818: URL: https://github.com/apache/iceberg/pull/17818
## Which Iceberg project does this PR belong to? Core ## What changes are proposed in this pull request? Fixes #17755. `RESTSessionCatalog.listTables`, `listNamespaces`, and `listViews` currently loop until the server returns a `null` `next-page-token`. If a REST server returns a token it has already returned earlier in the same call — whether the same token every time or an alternating cycle of two or more tokens — the client requests pages indefinitely and accumulates duplicated results. Every individual HTTP call succeeds, so socket timeouts do not bound the operation. The reporter observed 51,043 requests in 5 seconds against a stub that returned a fixed non-null token. This change adds a small `checkPageTokenNotRepeated` helper to `RESTSessionCatalog`. Each of the three listing loops tracks the set of `next-page-token` values it has received; on the second occurrence of the same token the client throws a `RESTException` with the offending token and the listing operation. A set (not just previous-token comparison) is required to also detect alternating-token cycles, per the possible-fix note in #17755. Returning a partial result would be unsafe because callers could treat missing namespaces, tables, or views as complete. Failing loudly matches the "favor the client" REST-spec guidance in `AGENTS.md` and the reporter's stated preferred behavior in the issue. ## Tests New unit tests added in `TestRESTCatalog` and `TestRESTViewCatalog`: - `listNamespacesFailsWhenServerRepeatsPageToken` — server returns the same non-null token every time. - `listNamespacesFailsWhenServerAlternatesPageTokens` — server alternates between two tokens; proves the guard uses a set, not just previous-token comparison. - `listTablesFailsWhenServerRepeatsPageToken`. - `listViewsFailsWhenServerRepeatsPageToke- `paginationTerminatesNormallyWhenServerReturnsNullToken` — regression: the guard does not affect the happy path when the server terminates pagination correctly. Existing `testPaginationForListNamespaces`, `testPaginationForListTables`, and `testPaginationForListViews` continue to pass unchanged. --- **AI Disclosure** - Platform/Tool: Cursor -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
