gabor-one opened a new issue, #1809: URL: https://github.com/apache/iceberg-python/issues/1809
### Apache Iceberg version 0.9.0 (latest release) ### Please describe the bug 🐞 Hi Everyone! I've encountered memory leak while using pyiceberg on Ubuntu aarch64 slim-container. There is no aarch64 wheel available so [pyiceberg/avro/decoder_fast.pyi](https://github.com/apache/iceberg-python/blob/main/pyiceberg/avro/decoder_fast.pyi#L20) is not built thus pyiceberg is reverting to native implementation. I'm getting the following warning. ``` /app/.venv/lib/python3.12/site-packages/pyiceberg/avro/decoder.py:185: UserWarning: Falling back to pure Python Avro decoder, missing Cython implementation warnings.warn("Falling back to pure Python Avro decoder, missing Cython implementation") ``` I'm writing pyarrow tables to AWS Glue repeatedly and it crashes with OOM after a couple of writes in the container. It works on macOS fine. ```python iceberg_table = self.glue_catalog.load_table( (self.database, self.table_name) ) iceberg_table.append(pa_table) ``` Tested in image: ghcr.io/astral-sh/uv:python3.12-bookworm-slim I'm trying to find out the reason for the leak. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [x] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
