ChangxingJiang opened a new issue, #2785:
URL: https://github.com/apache/iceberg-python/issues/2785
### Apache Iceberg version
None
### Please describe the bug đ
# Description
This error message does not appear consistently. In my case, it is reliably
triggered when performing concurrent queries on an Iceberg table, but it does
not occur when querying the same table with a single process or when querying
other tables.
This error occurs during the execution of weakref when the Python
interpreter's cleanup process is triggered.
The error message is as follows
```
Traceback (most recent call last):
File "/home/*/python3.10/site-packages/aiobotocore/session.py", line 36,
in __aexit__
await self._client.__aexit__(exc_type, exc_val, exc_tb)
File "/home/*/python3.10/site-packages/aiobotocore/client.py", line 644,
in __aexit__
await self._endpoint.http_session.__aexit__(exc_type, exc_val, exc_tb)
File "/home/*/python3.10/site-packages/aiobotocore/httpsession.py", line
109, in __aexit__
await self._exit_stack.aclose()
File "/home/*/python3.10/contextlib.py", line 656, in aclose
await self.__aexit__(None, None, None)
File "/home/*/python3.10/contextlib.py", line 714, in __aexit__
raise exc_details[1]
File "/home/*/python3.10/contextlib.py", line 697, in __aexit__
cb_suppress = await cb(*exc_details)
File "/home/*/python3.10/site-packages/aiohttp/client.py", line 1483, in
__aexit__
await self.close()
File "/home/*/python3.10/site-packages/aiohttp/client.py", line 1349, in
close
await self._connector.close()
File "/home/*/python3.10/site-packages/aiohttp/connector.py", line 1044,
in close
await super().close(abort_ssl=abort_ssl or self._ssl_shutdown_timeout ==
0)
RuntimeError: Task <Task pending name='Task-12'
coro=<ClientCreatorContext.__aexit__() running at
/home/*/python3.10/site-packages/aiobotocore/session.py:36>> got Future <Task
pending name='Task-13' coro=<_wait_for_close() running at
/home/*/python3.10/site-packages/aiohttp/connector.py:136>> attached to a
different loop
ERROR:asyncio:Task exception was never retrieved
future: <Task finished name='Task-12' coro=<ClientCreatorContext.__aexit__()
done, defined at /home/*/python3.10/site-packages/aiobotocore/session.py:35>
exception=RuntimeError("Task <Task pending name='Task-12'
coro=<ClientCreatorContext.__aexit__() running at
/home/*/python3.10/site-packages/aiobotocore/session.py:36>> got Future <Task
pending name='Task-13' coro=<_wait_for_close() running at
/home/*/python3.10/site-packages/aiohttp/connector.py:136>> attached to a
different loop")>
```
# Reason
This bug maybe be caused by #2495 . I revert #2495 and it can work normally.
In #2495 :
In FsspecFileIO, the S3FileSystem instanceâwhich carries an event loopâis
stored in threading.local(). This very likely (though I currently lack concrete
evidence; it may be caused in combination with other logic) leads to the error
during garbage collection at program shutdown.
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
