lintingbin commented on PR #4188: URL: https://github.com/apache/amoro/pull/4188#issuecomment-4279377360
> @lintingbin, @zhangwl9: I don’t think the RowGroup‑merging feature will make it into the Iceberg codebase. The core issue is that while this approach merges small files and reduces the file count, it does not reduce the number of RowGroups—they remain unchanged. As a result, the performance penalty of scanning Parquet files with many RowGroups still exists, so this type of compaction doesn’t really solve the underlying problem. So can we add additional parameter thresholds? For example, use raw-group merging only for files larger than a certain MB size, while smaller files continue to use the old merging method? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
