fallintoplace opened a new pull request, #2130: URL: https://github.com/apache/iceberg-go/pull/2130
**What** - Speed up nested variant column paths. **Why** - Building the path copies the growing slice at every level. - At 64 levels, path building is **88% faster**, with **127 -> 7 allocations**. - Full scan translation is **25% faster** at 64 levels. <details> <summary>Full benchmark results (13 cases)</summary> Apple M1 Pro, Go 1.26.3, one CPU. Medians of 7 paired runs at 1s per case, alternating base/head order. Both revisions used the same fixtures. Schema indexes were warmed before timing. | Case | ns/op before -> after | B/op before -> after | Allocs/op before -> after | | --- | ---: | ---: | ---: | | Path/depth=1 | 38.01 -> 38.02 | 16 -> 16 | 1 -> 1 | | Path/depth=2 | 105.1 -> 86.07 | 64 -> 48 | 3 -> 2 | | Path/depth=4 | 277.8 -> 155.5 | 208 -> 112 | 7 -> 3 | | Path/depth=8 | 689.3 -> 286.6 | 688 -> 240 | 15 -> 4 | | Path/depth=32 | 6734 -> 1414 | 9072 -> 1008 | 63 -> 6 | | Path/depth=64 | 20998 -> 2621 | 36272 -> 2160 | 127 -> 7 | | Path/missing | 10.41 -> 10.65 | 0 -> 0 | 0 -> 0 | | Scan translation/depth=1 | 4464 -> 4400 | 960 -> 960 | 17 -> 17 | | Scan translation/depth=2 | 4799 -> 4658 | 1128 -> 1112 | 21 -> 20 | | Scan translation/depth=4 | 5500 -> 5552 | 1640 -> 1544 | 29 -> 25 | | Scan translation/depth=8 | 7489 -> 7056 | 3192 -> 2744 | 45 -> 34 | | Scan translation/depth=32 | 25372 -> 20282 | 27925 -> 19852 | 141 -> 84 | | Scan translation/depth=64 | 69905 -> 52302 | 104100 -> 69905 | 269 -> 149 | Nested path cases are faster (p <= 0.004). Scan translation improves at depths 32 and 64 (p <= 0.004). The other cases show no significant time change (p >= 0.128). Run this 7 times per revision, alternating revisions: ```sh go test . -run '^$' -bench '^(BenchmarkColumnPathSegments|BenchmarkTranslateNestedVariantExtract)$' -benchmem -benchtime=1s -count=1 -cpu=1 ``` Base: `0b0828bffd8ba7f50e18cbafa59d6a4b8e5df2f6` Head: `1125226216d504869b8b1f96414050cf780578eb` </details> **Implementation** - Append the field names, then reverse the path once. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
