NoahKusaba opened a new pull request, #48: URL: https://github.com/apache/datafusion-iceberg/pull/48
## Which issue does this PR close? - Part of the Ballista-Iceberg integration (follows #20). ## Rationale for this change A distributed engine like Ballista serializes a catalog-backed `IcebergTableProvider` on one process and rebuilds it on another, such as its scheduler. #20 exposes the provider's parts (`catalog()`, `table_ident()` and `schema()`), but the only public constructor, `try_new`, reloads the table and takes the schema the table has now. If the table's schema has changed since the original provider was built, the rebuilt provider has a different schema and plans differently. ## What changes are included in this PR? - Adds `IcebergTableProvider::new_with_schema(catalog, table_ident, schema)`. It builds a provider from those accessors' return values without loading the table, so the rebuilt provider plans exactly as the original. - `try_new` now loads the schema and delegates to `new_with_schema`, and its docs gain an `# Errors` section. ## Are there any user-facing changes? One new public constructor. Nothing breaks. ## Are these changes tested? - `test_provider_rebuilt_with_schema_plans_like_the_original` adds a column after the original provider is built. It then checks that both the original and the rebuilt provider return the full, unchanged `SELECT *` output, without the new column. - A doctest rebuilds a provider from the original's accessors. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
