wgtmac opened a new pull request, #1115:
URL: https://github.com/apache/iceberg-cpp/pull/1115

   ### Problem
   
   The merging snapshot pipeline differs from Java in manifest pruning, retry 
state, delete validation, and execution order. For example, combining delete 
manifests from multiple snapshots into one index cannot represent successive 
deletion vectors for the same data file. Repeated application of copied append 
manifests can also lose or duplicate summary counts.
   
   ### Solution
   
   Align `MergingSnapshotUpdate`, `ManifestFilterManager`, and 
`ManifestMergeManager` with Java Iceberg at 
[0cb5041b9c7a](https://github.com/apache/iceberg/commit/0cb5041b9c7afc046e84406d2785ed79cd54fa46).
 Keep method order, branch order, and terminology close to the Java 
implementation to make comparison easier.
   
   - Track inherited manifest locations and use trusted references and 
partition summaries to prune filtering. Preserve cumulative delete identities, 
cache metrics evaluators by partition, and count duplicate deletes per manifest.
   - Match manifest grouping and `packEnd` ordering, first-manifest equality, 
retry caching, replaced-manifest accounting, and writer closure on failure.
   - Build delete indexes per snapshot, match DV conflict-validation shortcuts, 
and preserve copied-append summaries across repeated application and retry 
cleanup.
   - Match DV merge sequence inheritance and validate sequence/spec/partition 
compatibility. Use the configured executor for filtering, bin merging, and DV 
reads while preserving result order and stopping work after failure.
   - Add regression coverage for these behaviors, including null/NaN partition 
identity, partial failures, executor errors, and retry cleanup.
   
   Retain the existing C++ `Result`/`Status`, explicit executor, materialized 
reader, and transaction staging ownership contracts. This change does not 
introduce Java's encryption integration or event bus.
   
   ### Validation
   
   Rebased onto `apache/iceberg-cpp` main at 
`19573767858c16cb64a19e5c20afb303b8c6ada6` and rebuilt in Debug mode on macOS 
arm64, with tests and the bundled data libraries enabled 
(`ICEBERG_BUILD_REST=OFF`).
   
   - Full `schema_test`, `manifest_test`, and `table_update_test` suites 
passed: 1,876 tests passed and 48 were conditionally skipped (3/3 CTest 
targets).
   - The three changed public headers passed a C++20 compatibility-mode 
consumer syntax check.
   - `pre-commit` for all changed files and `git diff --check` passed.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to