roryqi commented on code in PR #10720: URL: https://github.com/apache/gravitino/pull/10720#discussion_r3077768621
########## design-docs/iceberg-supported-nested-namespace.md: ########## @@ -0,0 +1,116 @@ +# [Iceberg REST] Supported Nested Namespace Design + +## Background + +This document describes one practical solution to support Iceberg nested namespaces in Gravitino. +The scope is not only UI privilege granting, but also namespace mapping, identifier handling, +authorization scope, and compatibility behavior across Iceberg REST and Gravitino. + +References: + +- https://github.com/apache/gravitino/blob/main/docs/security/access-control.md +- https://github.com/apache/gravitino/blob/main/docs/iceberg-rest-service.md +- https://github.com/apache/gravitino/blob/main/docs/manage-relational-metadata-using-gravitino.md +- https://github.com/apache/gravitino/discussions/7296 + +## Goal + +- Support nested namespace operations from Iceberg REST to Gravitino through schema mapping. +- Support privilege granting for different nested namespace scopes (including UI workflow). +- Keep metadata model stable and avoid heavy refactor. + +## Non-Goal + +- Create nested namespace by Gravitino REST API directly. +- Delete nested namespace by Gravitino REST API directly. +- Modify nested namespace by Gravitino REST API directly (for example, rename or alter properties). +- Introduce a new metadata object (for example `SubNamespace`) in this phase. + +## Solution Options + +### Option A: Add a new metadata object `SubNamespace` + +Pros: + +- Clearer concept modeling. + +Cons: + +- Large refactor across metadata model, API, authorization, and UI. + +### Option B (Chosen): Use `Schema` with dot-separated name + +Pros: + +- Simple implementation. +- Reuses existing metadata and authorization model. +- In most cases, `schema` is enough for namespace management. +- Iceberg nested namespace is a special case, so avoiding a large refactor is more cost-effective. + +Cons: + +- Cannot naturally express nested tree structure in Gravitino storage. +- `list schema` remains flat (for example: `A`, `B`, `A.B`). + +## Design + +### Identifier Rules + +- Schema name can contain `.`. +- Add quote around schema names when needed to avoid parse ambiguity. +- Keep flat storage model and use dot-joined schema name as namespace mapping. +- `NameIdentifier` should support quoted `schema` names. +- In this phase, quoted identifier support is limited to `schema` only; other identifier parts keep existing parsing rules. + +Examples: + +- Nested namespace `A.B` maps to schema name `A.B`. +- Nested namespace `A.B.C` maps to schema name `A.B.C`. +- For parsing ambiguity, use quoted schema name in identifier rendering, for example: + - `metalake.catalog.'A.B'.table1` + - `metalake.catalog.'A.B.C'.table2` Review Comment: Invalid: https://example.com/path/to'O'reilly Valid (encoded): https://example.com/path/to%27O%27reilly We can encode it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
