technicolorbeat opened a new pull request, #2186: URL: https://github.com/apache/iceberg-go/pull/2186
## Summary - preserve deferred snapshot state when constructing MetadataBuilder from parsed commit metadata - represent builder additions and removals as an overlay over the immutable deferred base - keep lookup, add/ref/build, and remove/build paths lazy while retaining full materialization for collection access and serialization - add correctness coverage for cloning, defensive ownership, removal, remove/re-add, serialization, and v1/v2/v3 round trips - extend the REST benchmarks with add/ref/build, remove/build, and serialization cases Closes #1986. ## Design The parsed deferred snapshot state remains immutable from the builder perspective and is safe to share through its existing synchronized cache. Each builder owns a separately cloned overlay containing ordered additions and removed-ID tombstones. Snapshot membership uses the deferred ID index and does not decode snapshot bodies. Lookup checks eager additions first, then removal tombstones, then the deferred base. This ordering also preserves remove-then-readd semantics without duplicating or resurrecting the original snapshot. Build clones the overlay and the small eagerly known snapshot set before attaching them to the resulting metadata. Later builder mutations therefore cannot affect previously built metadata. Operations that require the complete ordered collection apply removals to the materialized base and append additions in builder order. The eager metadata path and public API are unchanged. ## Performance Apple M1 Max, medians of five runs from BenchmarkStageParseThenMetadataBuilder: | Metadata | Snapshots | Eager | Deferred | Time | Allocated bytes | Allocations | |---:|---:|---:|---:|---:|---:|---:| | 1,999,830 B | 3,418 | 81.16 ms | 52.61 ms | -35.2% | -44.2% | -79.1% | | 5,856,882 B | 10,000 | 236.74 ms | 147.09 ms | -37.9% | -44.5% | -79.2% | The issue originally measured the deferred path as 22-24% slower than eager at these sizes. Builder creation now preserves the parse-time benefit instead of immediately decoding historical snapshots. Representative add/ref/build and remove/build benchmarks also retain the lazy path. Full collection access and JSON serialization intentionally materialize all snapshots and are covered separately by the benchmark suite. ## Correctness The tests verify that: - builder construction and Build do not materialize deferred history - additions, ref updates, removals, and indexed lookups remain lazy - removed snapshots do not reappear during serialization - remove-then-readd replaces rather than duplicates a snapshot ID - builder clones and built metadata own independent mutable overlay state - complete history survives JSON serialization, equality, and v1/v2/v3 builder round trips - eager behavior remains unchanged ## Validation - go test ./... -count=1 - go vet ./... - go vet -tags=integration ./... - focused race-detector tests for deferred builder operations - golangci-lint v2.14.0: 0 issues - git diff --check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
