sungwy commented on issue #1148: URL: https://github.com/apache/iceberg-python/issues/1148#issuecomment-2338853251
Given that the catalog is instantiated through `load_catalog` entrypoint function, I think we'd actually need to introduce a catalog property as well, if we decide to introduce a new boolean flag to control this behavior. That being said, I'm still not convinced that this is necessary... I think keeping the approach of creating the tables on catalog initialization should result in the following: 1. if the tables exist, the DDL shouldn't be executed 2. if the tables don't exist, they are created 3. if the tables don't exist and the DDL fails to create the tables (e.g. because the DB doesn't allow creation of tables through standard DDLs), the DB should throw an error that should result in the program failing fast. > I really want it to fail fast if it does not exist instead of creating new tables and failing later, like when the system is live I'm not sure if I follow this point @isc-patrick . If we disable the creation of the table through the DDL, we will be failing at the same point, or even later. Since instead of failing at the Catalog initialization (for example by failing to create the tables), we will now be failing at a later part of the code that has assumed that the tables exist, and tries to execute one of the SQL queries for scans or commits. Sorry to probe - could you elaborate on how you expect the proposed approach of suppressing table creation would result in the application failing fast? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
