xpj01 opened a new issue, #1501: URL: https://github.com/apache/iceberg-python/issues/1501
### Apache Iceberg version 0.8.1 (latest release) ### Please describe the bug 🐞 I can load the glue catalog and list table from it, but got failed once load the table. I searched for the previous issues like https://github.com/apache/iceberg-python/issues/515 https://github.com/apache/iceberg-python/issues/892 I tired with those fixes but still not working for me. Here is my sample code. I tried to put all the potential properties into it. ``` catalog = load_catalog( "glue", **{ "type": "glue", "profile_name": "default", "glue.profile-name": "default", "glue.access-key-id": aws_credentials['aws_access_key_id'], "glue.secret-access-key": aws_credentials['aws_secret_access_key'], "glue.session-token": aws_credentials.get('aws_session_token', None), "glue.region": "us-west-2", "s3.access-key-id": aws_credentials['aws_access_key_id'], "s3.secret-access-key": aws_credentials['aws_secret_access_key'], "s3.session-token": aws_credentials.get('aws_session_token', None), "s3.region": "us-west-2", "aws_access_key_id": aws_credentials['aws_access_key_id'], "aws_secret_key_id": aws_credentials['aws_access_key_id'], "aws_secret_access_key": aws_credentials['aws_secret_access_key'], "aws_session_token": aws_credentials.get('aws_session_token', None), "region_name": "us-west-2", "warehouse": "s3://<bucket>/<warehouse folder>", "client.region": "us-west-2", "client.access-key-id": aws_credentials['aws_access_key_id'], "client.secret-access-key": aws_credentials['aws_secret_access_key'], "client.session-token": aws_credentials.get('aws_session_token', None) # Use get() in case token doesn't exist } ) ``` List tables works well for me ``` tables = catalog.list_tables("<db name>") for table in tables: print(table) ``` Load table doesn't work ` table = catalog.load_table("<db name>.<table name>")` trace > Traceback (most recent call last): File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 762, in _execute_task result = _execute_callable(context=context, **execute_callable_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/taskinstance.py", line 733, in _execute_callable return ExecutionCallableRunner( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/operator_helpers.py", line 252, in run return self.func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/models/baseoperator.py", line 422, in wrapper return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/python.py", line 238, in execute return_value = self.execute_callable() ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/operators/python.py", line 256, in execute_callable return runner.run(*self.op_args, **self.op_kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/operator_helpers.py", line 252, in run return self.func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/airflow/dags/iceberg_dag.py", line 61, in append_to_iceberg table = catalog.load_table("*****") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/pyiceberg/catalog/glue.py", line 472, in load_table return self._convert_glue_to_iceberg(self._get_glue_table(database_name=database_name, table_name=table_name)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/pyiceberg/catalog/glue.py", line 295, in _convert_glue_to_iceberg metadata = FromInputFile.table_metadata(file) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/pyiceberg/serializers.py", line 113, in table_metadata with input_file.open() as input_stream: ^^^^^^^^^^^^^^^^^ File "/home/airflow/.local/lib/python3.12/site-packages/pyiceberg/io/pyarrow.py", line 262, in open input_file = self._filesystem.open_input_file(self._path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pyarrow/_fs.pyx", line 789, in pyarrow._fs.FileSystem.open_input_file File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status OSError: When reading information for key 'iceberg_test/metadata/00004-804bd1cf-09b6-48b4-8d8b-4e332d971b13.metadata.json' in bucket '***': AWS Error ACCESS_DENIED during HeadObject operation: No response body. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [X] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
