danielcweeks commented on PR #9884: URL: https://github.com/apache/iceberg/pull/9884#issuecomment-1986761215
> I don't understand why we have s3 and s3a in ResolvingFileIO, actually why do we have ResolvingFileIO ? This is a good question and somewhat gets to the point. ResolvingFileIO was a way for users to be able to avoid having to lookup and set the FileIO implementation manually. The reason for the different schemes (e.g. `s3a`) is that people transitioning from Hive/Hadoop frequently used those legacy prefixes and ran into problems with mixing the more common `s3` scheme. Eventually this replaced the HadoopFileIO as the recommended FileIO since the native paths were more performant and tailored to Iceberg. So, if you're already using a custom FileIO, you need to set it one way or another. Now we're creating a second way to set it, and you have to set the scheme, which is probably already handled by the custom FileIO. (See the Dell EcsFileIO as an example). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
