zhangwl9 opened a new pull request, #4188:
URL: https://github.com/apache/amoro/pull/4188
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://amoro.apache.org/how-to-contribute/
2. If the PR is related to an issue in
https://github.com/apache/amoro/issues, add '[AMORO-XXXX]' in your PR title,
e.g., '[AMORO-XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][AMORO-XXXX] Your PR title ...'.
-->
## Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about its use case.
2. If you fix a bug, you can clarify why it is a bug.
3. Use Fix/Resolve/Close #{ISSUE_NUMBER} to link this PR to its related
issue
-->
Close #4186 .
## Brief change log
<!--
Clearly describe the changes made in modules, classes, methods, etc.
-->
1. **`ParquetFileMergeRunner`**
- Initializes `ParquetFileWriter` with target row group size
- Appends input files via `ParquetFileWriter.appendFile()` (zero-copy
row-group merging)
- Builds Iceberg `DataFile` with correct metrics via
`ParquetUtil.footerMetrics()`
- Handles failure cleanup (deletes partial output on error)
2. **`IcebergRewriteExecutor`** **enhancements:**
- `canParquetRowGroupMerge()` precondition checker validates:
- Table has no sort order
- All input files are Parquet format
- No encrypted data files
- No equality delete or position delete files
- No bloom filters enabled
- All files match current partition spec
- All input files have consistent Parquet schemas
- `parquetRowGroupMergeFiles()` merges files respecting
`self-optimizing.target-size` boundaries
- Falls back to row-based rewrite if any precondition fails or merge
throws
3. **`ParquetIOBridge`** — Package-scoped bridge in
`org.apache.iceberg.parquet` that exposes Iceberg's `ParquetIO.file()`
conversions (package-private) for the merge path.
4. **Configuration:**
- `self-optimizing.rewrite.use-parquet-row-group-merge.enabled` (default:
`false`)
## How was this patch tested?
- [x] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] Run test locally before making a pull request
## Documentation
- Does this pull request introduce a new feature? (yes / no)
- If yes, how is the feature documented? (not applicable / docs / JavaDocs /
not documented)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
