deepak-2605 opened a new issue, #1825: URL: https://github.com/apache/iceberg-go/issues/1825
### Apache Iceberg version main (development) ### Please describe the bug 🐞 TestPositionDeletePartitionedFanoutWriterEarlyStopCancelsRecordProduction (table/pos_delete_partitioned_fanout_writer_test.go) fails consistently on main. Arrow's CheckedAllocator catches unreleased allocations when a caller breaks out of positionDeletePartitionedFanoutWriter.Write() early. **Expected**: breaking out of Write()'s result iterator after the first data file should (a) stop the upstream record producer promptly, and (b) release every Arrow allocation made along the way — mem.CurrentAlloc() should be 0. **Actual**: the checked allocator reports stranded allocations every time. Steps to reproduce: `git clone https://github.com/apache/iceberg-go.git cd iceberg-go go test ./table/ -run TestPositionDeletePartitionedFanoutWriterEarlyStopCancelsRecordProduction -count=5 -v` Fails 5/5 runs. The leaked-byte total differs on every run (108160, 104640, 184320, 291840, 32320 bytes across five local runs), which points to a timing race rather than a deterministic leak , the amount stranded depends on how many batches happen to be in flight at the moment cancellation fires. **Stack trace (representative, from one run):** checked_allocator.go:186: LEAK of 64 bytes FROM github.com/apache/arrow-go/v18/arrow/array.(*bufferBuilder).resize+b0 .../arrow-go/[email protected]/arrow/array/bufferbuilder.go:92 checked_allocator.go:186: LEAK of 64 bytes FROM github.com/apache/arrow-go/v18/arrow/array.(*builder).init+c0 .../arrow-go/[email protected]/arrow/array/builder.go:145 checked_allocator.go:186: LEAK of 64 bytes FROM github.com/apache/arrow-go/v18/arrow/array.(*Int64Builder).newData+2f .../arrow-go/[email protected]/arrow/array/numericbuilder.gen.go:183 panic.go:694: invalid memory size exp=0, got=40320 --- FAIL: TestPositionDeletePartitionedFanoutWriterEarlyStopCancelsRecordProduction (0.02s) **Suspected area**: [#1595](https://github.com/apache/iceberg-go/pull/1595) hardened the shared startRecordFeeder/yieldDataFiles plumbing in partitioned_fanout_writer.go against this exact class of bug (retaining/releasing batches left in inputRecordsCh on early cancellation) and added this test as a regression guard. The leak appears to live downstream of that fix, in the position-delete-specific path ,either processBatch's handoff into RollingDataWriter.Add (pos_delete_partitioned_fanout_writer.go:126), or in how RollingDataWriter.stream's per-writer recordCh and bootstrap buffer are torn down when abortAll() (cancellation) rather than closeAll() (clean finish) stops the pipeline. That path is structurally separate from the channel [#1595](https://github.com/apache/iceberg-go/pull/1595) fixed, so it looks like it was never covered by the earlier hardening. Will be working on a fix for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
