corleyma commented on issue #17: URL: https://github.com/apache/iceberg-python/issues/17#issuecomment-1970530722
@Fokko We do a setup similar to this for integration tests, but the ability to write faster unit tests that depend only on a temp directory fixture in pytest has been great for our PySpark code. We had separately been using an InMemoryCatalog for unit tests of certain pyiceberg code, but now that we have more functions comingling pyspark and pyiceberg (ddl/metadata manipulation in pyiceberg), we are running into the limits of pyiceberg not supporting Hadoop catalog. I would love if we could add a file system catalog to PyIceberg compatible with PySpark and HadoopCatalog. It could be named and documented whichever way is needed to ensure folks know it's not a production catalog, but I think it's legitimately useful for testing purposes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
