isc-patrick commented on issue #1148: URL: https://github.com/apache/iceberg-python/issues/1148#issuecomment-2353288492
The problem I see with moving into the direction of creating integration tests for specific DB's(outside of SQLite) is that you are opening up to an enormous amount of possible tests. Postgres has dozens of releases over the past few years, https://www.postgresql.org/docs/release/, and SQLAlchemy supports 6 different Postgres drivers, https://docs.sqlalchemy.org/en/13/dialects/postgresql.html. Which combination of the 100's of possible ones are the integration tests going to cover? How many docker containers will be needed to be downloaded to run the int tests? And all of this to prove that the most ubiquitous OS SQL DB and Python ORM work with the 2 table SQLCatalog schema? I would skip the integration tests with Postgres and make some statements about assumptions for SQL databases working with pyiceberg via SQLAlchemy. SQLAlchemy is a well established abstraction layer and the SQLCatalog is extremely simple. Having followed this repo and the Slack for the last couple weeks, I know how hard the main commiters are working on this project and how much work there still is to bring this library up to speed with the other implementations. All of these tests add some amount of drag to a library - maintaining them, running them before commits, and most of all setting a precedent for the next set of developments like adding support for other SQL backends. I'm happy to help execute either way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
