roryqi commented on code in PR #10724: URL: https://github.com/apache/gravitino/pull/10724#discussion_r3055980122
########## design-docs/gravitino-logical-view-management.md: ########## @@ -0,0 +1,1051 @@ +# Design of Logical View Management in Gravitino + +## Background + +In modern data lakehouse architectures, views serve as a fundamental abstraction for data access, security enforcement, and query simplification. Organizations leverage multiple query engines (Trino, Spark, Hive) to access the same underlying data, but view management across these heterogeneous systems presents significant challenges: + +- **Portability Gap**: A view created in Trino cannot be read by Spark, and vice versa, due to differences in SQL dialects and metadata storage formats. +- **Fragmented Governance**: Views are scattered across different metastores (HMS, Iceberg REST Catalog, engine-specific stores), making unified access control and auditing difficult. +- **Inconsistent Security**: Each engine implements its own security model (definer/invoker), leading to inconsistent access control behavior across the data platform. + +Apache Gravitino, as a unified metadata management system, is well-positioned to address these challenges by providing centralized view management with multi-engine compatibility. + +--- + +## Goals + +1. **Multi-Engine Compatibility**: Views managed by Gravitino are visible and manageable across engines. Multi-dialect SQL representation storage enables cross-engine view sharing. + +2. **Unified View Management**: Provide standard CRUD operations for views: + - Create view + - Get/List views + - Alter view (update SQL, add representations, modify properties) Review Comment: Iceberg use the word `replaceView`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
