kevinjqliu commented on issue #1463: URL: https://github.com/apache/iceberg-python/issues/1463#issuecomment-2568401155
pyiceberg passes those s3 configs to the underlying filesystem for [pyarrow we use pyarrow.fs.S3FileSystem](https://github.com/apache/iceberg-python/blob/main/pyiceberg/io/pyarrow.py#L354-L380) for [fsspec we use s3fs.S3FileSystem](https://github.com/apache/iceberg-python/blob/main/pyiceberg/io/fsspec.py#L133-L168) I think your question is mostly related to 1. how to get different permissions from aws 2. how to pass those permissions to pyiceberg/filesystem #2 can be found in https://s3fs.readthedocs.io/en/latest/api.html#s3fs.core.S3FileSystem https://arrow.apache.org/docs/python/generated/pyarrow.fs.S3FileSystem.html #1 im not sure what the best practice is from AWS, but im sure there are tons of resources online about it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
