huaxingao opened a new pull request, #14954: URL: https://github.com/apache/iceberg/pull/14954
This PR adds Stage 1 support for Iceberg-managed SQL scalar UDFs in Spark 4.1, backed by the Iceberg REST catalog (list/load endpoints): - read-only scope: list + load + execute only (no CREATE/REPLACE/DROP). - Execution model: Spark executes by loading the SQL body/spec and using the analyzer rewrite, not session-scoped temp registration. - Out of scope for now: named args, default args, UDTF/TVF, full CRUD. Follow-up PRs: - write support stopgap, before Spark SPI: Spark procedure-based DDL stopgap (create/replace/drop via procedures) that calls REST catalog CRUD endpoints once they exist. - REST catalog CRUD + governance: REST catalog CREATE/REPLACE/DROP endpoints + request/response types + error mapping + Concurrency/idempotency semantics (ETag / If-Match, Idempotency-Key behavior), and authorization docs/tests. - after Spark lands CodeLiteralFunctionCatalog: Implement Spark’s CodeLiteralFunctionCatalog against the REST catalog endpoints -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
