robert3005 opened a new issue, #13567: URL: https://github.com/apache/iceberg/issues/13567
### Apache Iceberg version 1.9.1 (latest release) ### Query engine Spark ### Please describe the bug 🐞 BatchDataReader#open returns a CloseableIterator, however, this iterator is obtained from BaseBatchReader#newBatchIterable https://github.com/apache/iceberg/blob/main/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/BatchDataReader.java#L108-L116. The created CloseableIterable is immediately discarded and never closed. Whoever extends BaseBatchReader has to make sure to override open or remember to never register any resources on iterable returned from newBatchIterable. You can see that OrcIterable suffers from this and will never close the underlying OrcFile from https://github.com/apache/iceberg/blob/main/orc/src/main/java/org/apache/iceberg/orc/OrcIterable.java#L83-L84. Parquet doesn't suffer from this issue since the ParquetReader is added both to the CloseableIterable and CloseableIterator that's returned. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
