vitalyisaev2 opened a new issue, #42:
URL: https://github.com/apache/arrow-go/issues/42

   ### Describe the usage question you have. Please include as many useful 
details as  possible.
   
   
   I'm working on a streaming service that accumulates blocks of Arrow data, 
dumps them into Arrow Streaming IPC format and then writes them into network 
socket. The data is accumulated in `[]array.Builder`, which converts into 
`[]arrow.Array`, then into a single `arrow.Record` and finally into 
`bytes.Buffer`. 
   
   The key part of this algorithm is the separation of a new block: we need to 
know when the previous block is large enough (but not too large - because it 
could exceed message size limits in this case) to let us stop data accumulation 
in `[]array.Builder`, serialize existing data into `bytes.Buffer` and send over 
the network.
   
   So is it possible to estimate the number of bytes accumulated physically in 
`array.Builder`? Or may be it even possible to forecast the size of a future 
`bytes.Buffer` containing serialized data  from  `[]array.Builder`?
   
   ### Component(s)
   
   Go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to