fallintoplace opened a new pull request, #1890: URL: https://github.com/apache/iceberg-go/pull/1890
## What - Reuse metadata that stays constant for one position-delete file while emitting position-delete rows. - Prepare the partition, spec ID, delete-file path, and v3 content metadata once per file. - Keep the public DataFile getters unchanged. - Copy binary partition values before retaining the prepared metadata. ## Why For deletion vectors with many positions, the same partition remapping and metadata getter work was repeated for every output row. This moves that work from per-row to per-file. ## Checks - go test ./... - go test -race ./table - go test -tags=assert -v -run="^(TestInspectFilesTablesEarlyRelease|TestInspectDataFilesEmitsEmptyBatchWhenAllEntriesAreDeleted|TestInspectDataFilesEmptyTableEarlyRelease|TestInspectPositionDeletesEarlyRelease)$" ./table - go vet ./table ## Benchmark Compared prepare-per-row with prepare-once in the focused appender benchmark: - 100k rows, 1 partition field: 34.1 ms / 52.0 MB / 600k allocs -> 14.8 ms / 24.8 MB / 200k allocs - 100k rows, 5 partition fields: 75.7 ms / 77.1 MB / 1.4M allocs -> 49.9 ms / 49.9 MB / 1.0M allocs The benchmark is in table/inspect_position_deletes_bench_test.go. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
