fallintoplace opened a new pull request, #1917: URL: https://github.com/apache/iceberg-go/pull/1917
## What changed - **Deduplicate repeated partitions** while building equality-delete conflict filters. - **Reuse partition spec info** instead of resolving it for every delete file. - **Reuse the current schema and source-field lookups** during one filter build. - **Return the conservative `AlwaysTrue` path early** for non-identity transforms. - Add regression tests and a before/after benchmark. ## Why `eqDeletePartitionsToFilter` builds one row-space predicate for each equality-delete file. Multiple files can point to the same `(spec ID, partition)` tuple, so the same work gets repeated. This keeps the same conflict semantics but builds one term per distinct identity partition. ## Benchmark Command: ```text go test ./table -run '^$' -bench '^BenchmarkEqDeletePartitionsToFilter$' -benchmem -benchtime=200ms -count=5 ``` Machine: Apple M1 Pro, darwin/arm64 Median of 5 runs. Values are ns/op, B/op, allocs/op: | case | before | after | | --- | ---: | ---: | | fields=1, files=1,000, partitions=100 | 1,688,863 / 2,256,360 / 27,000 | 262,357 / 245,456 / 5,333 | | fields=1, files=10,000, partitions=100 | 14,162,586 / 22,563,829 / 270,000 | 2,602,603 / 2,215,171 / 49,461 | | fields=4, files=10,000, partitions=100 | 60,167,771 / 102,883,812 / 940,000 | 4,549,333 / 4,416,563 / 61,773 | | fields=1, files=1,000, partitions=1,000 | 1,435,163 / 2,256,359 / 27,000 | 379,229 / 310,256 / 8,123 | ## Tests - `go test ./table -count=1` - `go test ./table -race -count=1` - `go test ./... -count=1` - `go vet ./...` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
