dannypage commented on issue #1335: URL: https://github.com/apache/iceberg-python/issues/1335#issuecomment-2936111497
Hi folks! We are loving Iceberg and PyIceberg makes it a lot more accessible. We are currently doing a massive backfill (50k files per table) and seeing ~100-500 files per minute in S3 when working with batches of 1000 files. Going to test with the nightly build to see if there will be a performance, but I was curious about two things: - Will this Issue be included in the `0.10` release? - Is there a science to the ideal batch size for `add_files`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
