kevinjqliu commented on issue #1305: URL: https://github.com/apache/iceberg-python/issues/1305#issuecomment-2466466411
> pyiceberg.exceptions.CommitFailedException This is not a thread issue. This is expected when you have concurrent writers. The writer should retry the commit. See #1084 for a similar issue. And #269 as the issue to track this feature. See https://github.com/apache/iceberg-python/issues/269#issuecomment-2402969232 for a workaround. > i have too many rows to write to the same iceberg table.... how to speed it up? You can try writing batches as parquet files, and then collect them all in the main method. Use the `add_files` method to commit all parquet files to the iceberg table, this will commit only once to the table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
