heguanhui opened a new pull request, #51376: URL: https://github.com/apache/doris/pull/51376
### What problem does this PR solve? This feature addresses the issue that Doris cannot query Iceberg views. 1. Prerequisite Dependencies Iceberg version 1.7.x+ 2. Design Approach - Enhanced View Loading in IcebergMetadataCache: Added view loading capability during loadTable operations for Iceberg. Introduced getIcebergView() method to retrieve Iceberg views. - New Methods in IcebergExternalTable: isView(): Determines if the current Iceberg table is a view. getViewText(): Retrieves the SQL definition query of the view. - Execution Plan Generation Adjustment: Checks isView() during plan generation to identify Iceberg views. For views, generates a logical subquery execution plan by flattening the view into a subquery. - Cache Invalidation Enhancement: Added view cache invalidation in IcebergMetadataCache when invalidating catalog/database/table caches. ### Problem Summary: This PR aims to resolve the issue where querying an Iceberg view in Doris results in an error indicating the table does not exist. Specifically, it addresses the problem that Doris currently cannot recognize and query Iceberg views, causing exceptions when users attempt to access view data. ### Check List (For Author) Test (At least one of them must be included): Manual test scripts and report as below: [回归测试用例.docx](https://github.com/user-attachments/files/20514382/default.docx) ### Behavior changed: Yes. - BindRelation.java Modified getLogicalPlan method: Adjusted logic for Iceberg external tables to generate subquery execution plans when encountering views. Renamed parseAndAnalyzeHiveView to parseAndAnalyzeExternalView: Unified handling for external views (Hive/Iceberg). - Config.java **(1)** Added enable_query_iceberg_views: Configuration flag to enable/disable Iceberg view query support. - IcebergExternalTable.java **(1)** Enhanced initSchema method: Added schema initialization compatibility for both Iceberg tables and views. **(2)** New getViewText method: Retrieves the SQL definition of an Iceberg view. - IcebergMetadataCache.java **(1)** Constructor Initialization: Added cache loading for Iceberg views. **(2)** Invalidation Methods: Extended invalidateCatalogCache, invalidateTableCache, invalidateDbCache to include view cache invalidation. New Methods: **(1)** loadView: Loads Iceberg view metadata. **(2)** getIcebergView: Retrieves cached Iceberg view information. - IcebergMetadataOps.java **(1)** Modified listTableNames: Returns combined list of tables and views. New Methods: **(1)** viewExists: Checks if an Iceberg view exists. **(2)** loadView: Loads Iceberg view metadata. - IcebergUtils.java Modified Methods: **(1)** getSchema: Supports schema retrieval for both tables and views. **(2)** loadSchemaCacheValue: Handles schema caching for both types. New Methods: **(1)** getIcebergView/getIcebergViewInternal: Retrieve Iceberg view objects. **(2)** getConvertedSchema: Converts Iceberg schema to Doris-compatible format. - ExternalMetadataOps.java New Interface Methods: **(1)** loadView: Load external view metadata. **(2)** - viewExists: Check view existence. **(3)** listViewNames: List available views. - IcebergExternalCatalog.java **(1)** Modified listTableNames: Returns combined tables and views. **(2)** Overridden viewExists: Implements view existence check. - ExternalCatalog.java **(1)** New Interface Method: viewExists to check view existence. - ExternalTable.java New Methods: **(1)** isTable: Determines if the object is a table (vs. view). **(2)** getRemoteDbName: Retrieves the remote database name. - HMSExternalTable.java Modified Methods: **(1)** initSchema: Added view compatibility. **(2)** getIcebergSchema: Supports both tables and views. **(3)** Overridden isTable: Implements table type check. - IcebergApiSource.java **(1)** Constructor Adjustment: Added view type validation to prevent invalid operations. - IcebergTableSink.java **(1)** Constructor Adjustment: Added view type validation to prevent invalid operations. Does this need documentation? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
