smaheshwar-pltr commented on issue #1589: URL: https://github.com/apache/iceberg-python/issues/1589#issuecomment-2633662660
Thanks @bigluck @kevinjqliu for this valuable discussion. Just pointing out that IMO REST catalogs can righly be reliant on clients (generally) prioritising the IO returned in the `table_response.config`. For things like federation, I think the Catalog returning the appropriate IO and clients adhering is crucial. In PyIceberg, the codepath that this depends on is (as already mentioned) https://github.com/apache/iceberg-python/blob/90706ea8bd335dda6af08cf32dc5b4cb9ecf259c/pyiceberg/catalog/rest.py#L527 and in the Java implementation, it's https://github.com/apache/iceberg/blob/1afa0e739b62355518942629b30c845e51e724a6/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L1038. I do think that *PyIceberg having consistent behaviour with the Java implementation* here is a big advantage. @bigluck's idea of explicitly allowing clients to override that behaviour if they wish (so if stuff goes wrong it's on them) seems very reasonable to me 😄. IMHO the interface to this should indicate that this is exceptional behaviour. (Also, my sense is that we'd want to involve other implementations in this discussion via a thread but I'll let @kevinjqliu express his thoughts on this matter and that action first.) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
