amogh-jahagirdar commented on issue #8953: URL: https://github.com/apache/iceberg/issues/8953#issuecomment-1805001659
Looking at the code, this shouldn't happen but would need to check more deeply. We create an `OutputFileFactory` per writer, https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java#L667. Every `OutputFileFactory` should generate a unique file name, https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/io/OutputFileFactory.java#L99 the relevant part here being the last part where it's a monotonically increasing file number. The only way I can see this maybe happening is if there are somehow multiple OutputFileFactory's happening concurrently (thus multiple concurrent writers in the same task, which I don't think can happen. but that's the part i'll need to double check). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
