mrcnc commented on PR #11294: URL: https://github.com/apache/iceberg/pull/11294#issuecomment-2420675717
> Hi @mrcnc, @RussellSpitzer . To confirm this PR solution to 10127: any existing (or new) iceberg tables stored in azure with wasbs + .blob. will be interpreted interchangeably with abfss prefix + .dfs. endpoint, regardless of whether the table's existing metadata displays wasbs + .blob. vs the new abfss + dfs? > That being said, does this mean iceberg api supports tables with possibly mixed schemes? (i.e. some paths in a table's metadata were written with 'wasbs' while others were with 'abfss')? The path for new files in existing tables is determined by the table's base location or [LocationProvider](https://iceberg.apache.org/javadoc/latest/org/apache/iceberg/io/LocationProvider.html) so tables that have paths using the `wasbs` scheme would still use the same scheme for new files. For new tables, the location will default to the default location for the warehouse or namespace but each table can explicitly provide a location when it is created. It seems possible that there could be mixed schemes (for example you changed the implementation of a custom LocationProvider) so I think it's worthwhile to understand if mixed paths are supported but I'm not sure currently. The intent of this PR was to use the ADLSFileIO for wasbs paths by default, instead of delegating these to HadoopFileIO as a fallback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
