pvary commented on PR #12824: URL: https://github.com/apache/iceberg/pull/12824#issuecomment-2838515470
@coderfender: Be aware that the planning part of the data file rewrite action is moved to core. There is a PR under review (#12692) which will move the Spark action to use the new core API. Also, could you please help me understand where the number of files becomes a bottleneck? We had very similar issues when working on the Flink rewrite data files. There we decide to limit the size of the full compaction (https://github.com/apache/iceberg/pull/11497/files#diff-9c2fb1c711a30af44656320fc301b42055ad4f2077c52010413eee4bf8deb854R60). We might have very similar issues here, with a slightly different solution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
