abhirathod95 commented on PR #886:
URL: https://github.com/apache/iceberg-go/pull/886#issuecomment-4239686292
Added BenchmarkFanoutMemory in table/fanout_memory_bench_test.go to measure
the improvement.
The benchmark uses a gated RecordReader that feeds 100 batches (50K rows
each) one at a time through a single fanout writer, measuring HeapInuse between
sends via runtime.GC() + runtime.ReadMemStats().
This directly observes whether previous batches are retained (old) or
released promptly (new).
```
benchstat (old vs new, n=6):
│ OLD (pre-fix) │ NEW (with fix)
│
│ peak-heap-delta │ peak-heap-delta vs base
│
FanoutMemory-14 2.111Gi ± 6% 1.593Gi ± 6% -24.53% (p=0.002)
│ sec/op │ sec/op vs base
│
FanoutMemory-14 1.555 ± 1% 1.550 ± 1% ~ (p=0.699)
│ B/op │ B/op vs base
│
FanoutMemory-14 19.48Gi ± 1% 19.59Gi ± 1% ~ (p=0.485)
│ allocs/op │ allocs/op vs base
│
FanoutMemory-14 36.31M ± 0% 36.31M ± 0% ~ (p=0.240)
```
~25% reduction in peak live heap (p=0.002), no change in wall-clock time
or total allocations
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
