coderfender commented on PR #12824: URL: https://github.com/apache/iceberg/pull/12824#issuecomment-2838760858
Sure @pvary ,the goal here is to provide users an option to limit number of files to rewrite which could potentially destabilize jobs in case the file count is super high (which is most common in large data platforms including the one I manage atm). Also, I plan to raise a similar PR on the flink side to support this option (discussed this with @mxm internally) in the future . However if you think we are closer in terms of merging spark rewrite refactor changes to main branch soon, I can wait on this changes, rebase code (although I am more inclined towards getting this merged first) and raise PR on new main branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
