roryqi commented on code in PR #10720: URL: https://github.com/apache/gravitino/pull/10720#discussion_r3080842742
########## design-docs/iceberg-supported-nested-namespace.md: ########## @@ -0,0 +1,129 @@ +# [Iceberg REST] Supported Nested Namespace Design + +## Background + +This document describes one practical solution to support Iceberg nested namespaces in Gravitino. +The scope is not only UI privilege granting, but also namespace mapping, identifier handling, +authorization scope, and compatibility behavior across Iceberg REST and Gravitino. + +References: + +- https://github.com/apache/gravitino/blob/main/docs/security/access-control.md +- https://github.com/apache/gravitino/blob/main/docs/iceberg-rest-service.md +- https://github.com/apache/gravitino/blob/main/docs/manage-relational-metadata-using-gravitino.md +- https://github.com/apache/gravitino/discussions/7296 + +## Goal + +- Support nested namespace operations from Iceberg REST to Gravitino through schema mapping. +- Support privilege granting for different nested namespace scopes (including UI workflow). +- Keep metadata model stable and avoid heavy refactor. + +## Non-Goal + +- Create nested namespace by Gravitino REST API directly. +- Delete nested namespace by Gravitino REST API directly. +- Modify nested namespace by Gravitino REST API directly (for example, rename or alter properties). +- Introduce a new metadata object (for example `SubNamespace`) in this phase. + +## Solution Options + +### Option A: Add a new metadata object `SubNamespace` + +Use a new metadata object `SubNamespace` to replace schema + +Catalog -> SubNamespace a -> SubNamespace a.b -> Table a.b.c + -> SubNamespace a.c -> SubNamespace a.c.d -> Table a.c.d.e + +Pros: + +- Clearer concept modeling. + +Cons: + +- Large refactor across metadata model, API, authorization, and UI. + +### Option B (Chosen): Use `Schema` with dot-separated name + +Pros: + +- Simple implementation. +- Reuses existing metadata and authorization model. +- In most cases, `schema` is enough for namespace management. +- Iceberg nested namespace is a special case, so avoiding a large refactor is more cost-effective. + +Cons: + +- Cannot naturally express nested tree structure in Gravitino storage. +- `list schema` remains flat (for example: `A`, `B`, `A.B`). + +## Design + +### Identifier Rules + +- Schema name can contain `.`. +- Add quote around schema names when needed to avoid parse ambiguity. +- Keep flat storage model and use dot-joined schema name as namespace mapping. +- `NameIdentifier` should support quoted `schema` names. +- In this phase, quoted identifier support is limited to `schema` only; other identifier parts keep existing parsing rules. Review Comment: Gravitino REST API only has list schema API, we won't list sub-schema. We have four namespaces. A.B.C, A, B and A.B. When listing schema, we will return all the schemas, A.B.C, A, B and A.B. We won't only return the first level namespaces A and B. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
