[I] [C++][Parquet] Loading big parquet files leads to huge memory consumption [arrow]

[via GitHub]

KuczaRacza opened a new issue, #44890:
URL: https://github.com/apache/arrow/issues/44890

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   Loading big parquet files (23GB file) leads to high memory consumption, 
ending in OOM. Memory consumption surpass my machine limit of 64GB ram.
   
   Reader is set up similar as  demonstrated on docs page via `` 
parquet::arrow::FileReaderBuilder`` with key difference of
    passing custom allocator. Huge usage of memory is not report by my custom 
allocator nor by memory pool passed to reader.  
   
   After invastigation iI encouter thath RangeReadeCache created lot of  cache 
entries, 735 to be exact witch each about 33MiB of length. Each with it's 
future with use count 1. 
   
![enteries](https://github.com/user-attachments/assets/e2eb65ab-8163-4bfd-b4cd-fc3dd2335f11)
   Also for some reason this uses default memory pool
   
![image](https://github.com/user-attachments/assets/95b78844-e590-4e67-8c89-45c0d440516b)
   
   
   ### Component(s)
   
   C++


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org