geruh commented on issue #515: URL: https://github.com/apache/iceberg-python/issues/515#issuecomment-2029207168
Your glue calls look, fine but your S3 calls are the problem. I was able to reproduce the issue by having the incorrect region for my AWS profile at `./aws/config` and passing in the region config upon initializing the catalog. aws_config ``` [test] region = us-east-1 ``` catalog init ``` catalog = pyiceberg.catalog.load_catalog( catalog_name, **{"type": "glue", "profile_name": "test", "region_name": "us-west-2"} ) ``` Which leads to this exception ``` File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status OSError: When reading information for key 'test/metadata/00000-c0fc4e45-d79d-41a1-ba92-a4122c09171c.metadata.json' in bucket 'test_bucket': AWS Error UNKNOWN (HTTP status 301) during HeadObject operation: No response body. ``` It looks like when we infer the correct FileIO the aws profile region config takes precedence over the catalogs region config. For now you should be able to work around this by ensuring the profiles region is in sync with the config passed into the catalog. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org