bk-mz commented on issue #9833: URL: https://github.com/apache/iceberg/issues/9833#issuecomment-1988535507
Just to give more overview, after something, iceberg has created a mirror of existing partitions in which it has put TONS of small files. Example: we have "phantom" partition `474970` that corresponds to Fri Mar 08 2024 10:00:00 GMT+0000. Both folders are present on s3, but the odd one `474970` contains 2,5k files totallin 250mb. So we see both partition with hour as plain text and with number like `474970`. `rewrite_position_delete_files` keeps processing those folder and rewriting datafiles to themselves, with each rewrite data is appended to "phantom" odd partitions, leading to table degradation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
