mchades commented on code in PR #10720: URL: https://github.com/apache/gravitino/pull/10720#discussion_r3152387231
########## design-docs/iceberg-supported-nested-namespace.md: ########## @@ -0,0 +1,582 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> +# [Iceberg REST] Supported Nested Namespace Design + +## Background + +This document describes one practical solution to support Iceberg nested namespaces in Gravitino. +The scope is not only UI privilege granting, but also namespace mapping, identifier handling, +authorization scope, and compatibility behavior across Iceberg REST and Gravitino. + +References: + +- https://github.com/apache/gravitino/blob/main/docs/security/access-control.md +- https://github.com/apache/gravitino/blob/main/docs/iceberg-rest-service.md +- https://github.com/apache/gravitino/blob/main/docs/manage-relational-metadata-using-gravitino.md +- https://github.com/apache/gravitino/discussions/7296 + +## Goal + +- Support nested namespace operations from Iceberg REST to Gravitino through schema mapping. +- Support privilege granting for different nested namespace scopes (including UI workflow). +- Keep metadata model stable and avoid heavy refactor. + + +## Solution Options + +### Option A: Add a new metadata object `NestedNamespace` + +Use a new metadata object `NestedNamespace` to represent nested namespace explicitly. +`NestedNamespace` has a one-to-one mapping with Iceberg `Namespace` to avoid ambiguity +with existing Gravitino `Namespace` concepts. + +Catalog -> NestedNamespace a -> NestedNamespace a.b -> Table a.b.c + -> NestedNamespace a.c -> NestedNamespace a.c.d -> Table a.c.d.e + +Pros: + +- Clearer concept modeling. + +Cons: + +- Large refactor across metadata model, API, authorization, and UI. + +### Option B (Recommended): Reuse `Schema` entity and enhance schema expression capability + +Keep physical metadata unchanged (still persisted as `Schema`) and introduce +`HierarchicalSchema` as a logical expression layer in Iceberg REST adaptation, +identifier rendering, and authorization scope matching. + +Pros: + +- Low-impact evolution path without introducing a new metadata entity. +- Decouples nested namespace semantics from `.` and reduces parser ambiguity. +- Reuses existing metadata and authorization model to reduce implementation risk. + +Cons: + +- Requires explicit conversion rules between logical path and physical schema name. +- Authorization matching and identifier serialization become more complex. +#### Option B Separator: Configurable logical separator (default `:`) + +Examples: + +- `A:B:C` as logical `HierarchicalSchema` path when separator is `:`. +- Physical schema name remains mapped through conversion layer. + +Pros: + +- Better readability than escaping `.` in many clients and UI forms. +- Lower routing conflict risk than `/`. +- Easier to keep backward compatibility with existing non-nested schema handling. + +Cons: + +- Needs clear validation rule to avoid ambiguity with existing schema names containing configured + separator. Review Comment: Is there a more detailed explanation for this part in this design document? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
