fallintoplace opened a new pull request, #2252: URL: https://github.com/apache/iceberg-go/pull/2252
## What Speed up ordering keys for the `partitions` metadata table. ## Why Each field was formatted with `fmt.Fprintf`, which adds cost for large and wide tables. ## Implementation - Format common primitive values directly. - Keep sorted field IDs and the exact `%T:%v` output. - Keep `fmt` for other types and add parity coverage. ## Benchmark Function-only benchmark with 1,000 prebuilt partition maps. Median of three 120 ms runs on Linux/amd64, Go 1.23.2, AMD EPYC 9V74. This does not measure metadata scans. | Value | Fields | ns/op before → after | B/op before → after | allocs/op before → after | | --- | ---: | ---: | ---: | ---: | | int32 | 1 | 281 → 153 | 64 → 64 | 4 → 3 | | string | 1 | 274 → 176 | 80 → 112 | 4 → 4 | | binary | 1 | 385 → 276 | 96 → 112 | 4 → 4 | | mixed | 1 | 264 → 161 | 64 → 64 | 4 → 3 | | int32 | 8 | 1,647 → 481 | 400 → 288 | 14 → 3 | | string | 8 | 1,432 → 602 | 640 → 672 | 14 → 4 | | binary | 8 | 2,520 → 1,540 | 880 → 672 | 14 → 4 | | mixed | 8 | 1,810 → 540 | 656 → 288 | 15 → 3 | | int32 | 32 | 7,703 → 3,182 | 1,552 → 1,056 | 40 → 3 | | string | 32 | 8,083 → 3,471 | 3,826 → 2,208 | 40 → 4 | | binary | 32 | 10,926 → 6,322 | 3,121 → 2,208 | 40 → 4 | | mixed | 32 | 8,406 → 3,835 | 2,448 → 1,056 | 41 → 3 | Some small string and binary cases use more bytes, but still run faster. The PR includes `BenchmarkInspectPartitionKey` to compare both implementations in the repository. ```sh go test ./table -run '^TestInspectPartitionKeyMatchesFmt$' -bench '^BenchmarkInspectPartitionKey$' -benchmem -benchtime=120ms -count=3 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
