talatuyarer opened a new pull request, #17859: URL: https://github.com/apache/iceberg/pull/17859
This PR adds the read path for Iceberg views to Flink, the first step of the plan in #17858 With this change, views created through the Iceberg `ViewCatalog` API or by other engines such as Spark become queryable from Flink SQL. Write operations such as `CREATE VIEW`, `DROP VIEW`, `ALTER VIEW ... RENAME` and `ALTER VIEW AS` are follow-ups tracked in #17858. ### Resolution semantics and limitations Flink's `Catalog` API gives an implementation no way to inject the resolution context that the Iceberg view spec defines, so two limitations are worth calling out: 1. **Stored `default-catalog`/`default-namespace` are not honored by the Flink planner.** The view spec says engines should resolve unqualified references against the defaults stored in the view version, and Spark does this natively via its `View#currentCatalog()/currentNamespace()` hooks. Flink has no equivalent. It always resolves unqualified references against the view's own catalog and database. For views whose `default-namespace` equals the namespace they live in (the common case, and what the follow-up `CREATE VIEW` PR will always produce, the two coincide and resolution is correct. A view created by another engine with a *different* `default-namespace` may resolve unqualified references differently in Flink than in Spark/Trino. Fixing this properly needs a Flink-side API; until then this is documented behavior. 2. **Lenient dialect fallback.** If a view has no `"flink"` SQL representation,`View#sqlFor` falls back to the closest available representation (e.g. Spark SQL). ANSI-compatible SQL works across engines (covered by tests), but non-portable SQL fails at parse time or, worse, could parse with different semantics. This is the same stance Spark takes today; a strict-dialect option is planned as a follow-up. See also the https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81 for the longer-term cross-engine story. 3. **Catalog names in qualified references are deployment-local.** If the stored view SQL fully qualifies references with a catalog name, that name only resolves in deployments that register the Iceberg catalog under the same name. This is inherent to the view spec storing engine-local catalog names and applies to Spark equally; unqualified references (resolved per item 1) do not have this problem. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
