gruuya commented on issue #840: URL: https://github.com/apache/iceberg-rust/issues/840#issuecomment-2561395335
> we need to add catalog support for datafusion Hmm, I think that is adequately supported already, no? https://github.com/apache/iceberg-rust/blob/5400bbee9697189bb95646f5e3c0eb9a84d4b376/crates/integrations/datafusion/src/catalog.rs#L86 https://github.com/apache/iceberg-rust/blob/5400bbee9697189bb95646f5e3c0eb9a84d4b376/crates/integrations/datafusion/src/schema.rs#L86 To be clear, i think the DF `CREATE EXTERNAL TABLE` construct (formalized via `TableProviderFactory`) is at most loosely coupled to a catalog. In fact the use case is typically just registering a pre-existing table in an ad-hoc manner for some read-only queries. To me this is analogous to FDWs in Postgres. While it is possible to wire-up the write path for those, it would require implementing `TableProvider::insert_into` in https://github.com/apache/iceberg-rust/blob/5400bbee9697189bb95646f5e3c0eb9a84d4b376/crates/integrations/datafusion/src/table/mod.rs#L113 (but I also think this is orthogonal to the ask here). Contrast that to a regular `CREATE TABLE` construct which would correspond to a full coupling with a catalog, targeting tables native to the given system. Thus a full life-cycle of the table would need to be tracked, but that is also beyond the scope of this issue. At least that is how we perceive/use them, would be curious to hear other interpretations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
