[I] Memory Leak about splitAndTransfer [arrow]

[via GitHub]

aqni opened a new issue, #41648:
URL: https://github.com/apache/arrow/issues/41648

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   version: 16.0.0
   
   I am currently working on a project using Apache Arrow in Java and I have 
encountered a potential memory leak issue.
   
   run the showMemoryLeak method
   
   ```java
     @Test
     public void showMemoryLeak() {
       RootAllocator allocator = new RootAllocator(Long.MAX_VALUE);
       ValueVector slice;
       try (BufferAllocator child = allocator.newChildAllocator("child", 0, 
Long.MAX_VALUE)) {
         try (IntVector vector = new IntVector("vector", child)) {
           vector.setSafe(0, 1);
           vector.setValueCount(1);
           TransferPair transferPair = vector.getTransferPair(allocator);
           transferPair.splitAndTransfer(0, 1);
           slice = transferPair.getTo();
         }
       }
       slice.close();
     }
   ```
   
   then a exception is thrown
   
   ```
   java.lang.IllegalStateException: Attempting operation on allocator when 
allocator is closed.
   Allocator(child) 0/0/16384/9223372036854775807 (res/actual/peak/limit)
   
   
        at 
org.apache.arrow.memory.BaseAllocator.assertOpen(BaseAllocator.java:172)
        at org.apache.arrow.memory.BufferLedger.decrement(BufferLedger.java:149)
        at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:125)
        at org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:105)
        at 
org.apache.arrow.vector.BaseValueVector.releaseBuffer(BaseValueVector.java:114)
        at 
org.apache.arrow.vector.BaseFixedWidthVector.clear(BaseFixedWidthVector.java:247)
        at 
org.apache.arrow.vector.BaseFixedWidthVector.close(BaseFixedWidthVector.java:238)
        at 
cn.edu.tsinghua.iginx.parquet.util.arrow.ArrowVectorsTest.showMemoryLeak(ArrowVectorsTest.java:30)
        ...
   ```
   
   If I continue to execute transfer() explicitly, there will be no memory 
leak, as shown below
   
   ```java
     @Test
     public void showMemoryLeak() {
       RootAllocator allocator = new RootAllocator(Long.MAX_VALUE);
       ValueVector slice;
       try (BufferAllocator child = allocator.newChildAllocator("child", 0, 
Long.MAX_VALUE)) {
         try (IntVector vector = new IntVector("vector", child)) {
           vector.setSafe(0, 1);
           vector.setValueCount(1);
           TransferPair transferPair = vector.getTransferPair(allocator);
           transferPair.splitAndTransfer(0, 1);
           try(ValueVector to = transferPair.getTo()) {
             TransferPair transferPair2 = to.getTransferPair(allocator);
             transferPair2.transfer();
             slice = transferPair2.getTo();
           }
         }
       }
       slice.close();
     }
   ```
   
   This problem is caused by the fact that when startIndex == 0, 
splitAndTransfer() does not transfer the validity buffer.
   
   ### Component(s)
   
   Java


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org