fallintoplace opened a new pull request, #1971: URL: https://github.com/apache/iceberg-go/pull/1971
## What - **Compute partition residuals during local planning.** - **Cache one evaluator per partition spec** and reuse it for every data file in that spec. - Use strict and inclusive projections to return `AlwaysTrue`, `AlwaysFalse`, or the remaining row filter for each task. - Cover identity, time, truncate, bucket, null, set, boolean, and transformed filter predicates. Unsupported transforms stay conservative. - **Read only the fields needed by each task residual**, so a fully satisfied partition filter does not add filter-only columns to the file read. ## Example For `ts >= ... AND ts < ...` on `day(ts)` partitions: - Interior partitions get `AlwaysTrue`. - Boundary partitions keep only the needed timestamp predicate. - Partitions outside the range get `AlwaysFalse`. ## Benchmark 4,096 files on an Apple M1 Pro: - **Identity residual evaluation:** about 0.67 ms/op, 360 KB/op, 16,384 allocs/op - **Day-range residual evaluation:** about 0.98 ms/op, 449 KB/op, 25,396 allocs/op ## Checks - `go test ./table/... -count=1` - `go test -race ./table -count=1` - `go test ./... -run '^$' -count=1`\n- `go vet ./...`\n- `go run github.com/golangci/golangci-lint/v2/cmd/[email protected] run --timeout=10m`\n\n🔥 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
