ngl-william-t opened a new issue, #3244:
URL: https://github.com/apache/iceberg-python/issues/3244
### Apache Iceberg version
None
### Please describe the bug 🐞
We have our datalake that is apache iceberg format that is governed by
lakeformation (account A). We have various other AWS accounts that are trying
to read in a lambda using pyiceberg.
All boto3 calls to fetch the data work correctly, to me verifying that the
IAM permissions and Lake formation permissions are correct. Additionally boto3
athena calls are successful. However when pyiceberg, it fails with the
following error:
`[ERROR] ForbiddenError: RESTError 403: Received unexpected JSON Payload:
{
"error": {
"code": 403,
"type": "AccessDeniedException"
}
}
, errors: Field required
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 38, in lambda_handler
results = fetch_dataset()
File "/var/task/lambda_function.py", line 64, in fetch_dataset
table = catalog.load_table("curated_sales_and_policy.ods_agent")
File "/opt/python/lib/python3.13/site-packages/tenacity/__init__.py", line
331, in wrapped_f
return copy(f, *args, **kw)
File "/opt/python/lib/python3.13/site-packages/tenacity/__init__.py", line
470, in __call__
do = self.iter(retry_state=retry_state)
File "/opt/python/lib/python3.13/site-packages/tenacity/__init__.py", line
371, in iter
result = action(retry_state)
File "/opt/python/lib/python3.13/site-packages/tenacity/__init__.py", line
393, in <lambda>
self._add_action_func(lambda rs: rs.outcome.result())
File "/var/lang/lib/python3.13/concurrent/futures/_base.py", line 449, in
result
return self.__get_result()
File "/var/lang/lib/python3.13/concurrent/futures/_base.py", line 401, in
__get_result
raise self._exception
File "/opt/python/lib/python3.13/site-packages/tenacity/__init__.py", line
473, in __call__
result = fn(*args, **kwargs)
File
"/opt/python/lib/python3.13/site-packages/pyiceberg/catalog/rest/__init__.py",
line 921, in load_table
_handle_non_200_response(exc, {404: NoSuchTableError})
File
"/opt/python/lib/python3.13/site-packages/pyiceberg/catalog/rest/response.py",
line 111, in _handle_non_200_response
raise exception(response) from exc`
here is the relevant piece of code:
` catalog = load_catalog(
"glue_rest",
**{
"type": "rest",
"uri": "https://glue.us-east-2.amazonaws.com/iceberg";,
"warehouse": "account_A_account_number",
"rest.sigv4-enabled": "true",
"rest.signing-region": "us-east-2",
"rest.signing-name": "glue",
}
)
table = catalog.load_table("database1.table1")`
here is our settings:
AccountA:
- lakeformation describe on Database
- lakeformation describe, select on relevant tables
- have a proper data location access resource create in lakeformation
granting the execution role the appropriate access
AccountB:
- lambda execution role has glue:* and lakeformation:GetDataAccess
Just to note, the catalog is created successfully and doesn't throw an
error, but it does when we specifically on load_table()
I've tried troubleshooting by playing with the headers being sent and
messing with the header value sig4-enabled. Nothing seems to work.
Does anyone have a working example and the configuration you are using? Is
this support with the current version (we are on latest 0.11.1).
Any help would be appreciated. I can provide additional information if
needed.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
