feefladder opened a new issue, #6647:
URL: https://github.com/apache/arrow-rs/issues/6647

   **Which part is this question about**
   Alignment of returned `Bytes` from `object_store.get_range()` or 
`object_store.get_ranges`
   
   
   
   **Describe your question**
   
   Are there any alignment guarantees for `get_range`, or would that be a 
feature that could be added?
   
   **Additional context**
   
   
   I want to use `object_store` as a possible io for a [crate I'm working 
on](https://github.com/feefladder/tiff2). Now, ideally, I'd be directly using 
the provided `Bytes` from the `get_range(s)` function. That needs two things:
   1. Alignment of the result is >= alignment of the underlying type (8-byte at 
most)
   2. We can convert to a `BytesMut` e.g. Does `object_store` hold any other 
references to the byte struct?
   
   
[this](https://github.com/apache/arrow-rs/blob/7bcc1ad988498def843180c6a4c95f9732f31a4b/object_store/src/util.rs#L53)
 seems to be the relevant code here. Bytes [tests for 
misalignments](https://github.com/tokio-rs/bytes/pull/346/files#diff-59a7c57d16e92b38766e932c25b74855e4be427436d609fbd2e0a7592284eac9)
 
   
   I know the upstream issues (tokio-rs/bytes/#437 tokio-rs/bytes/#343) and 
that [zero-copy works for 
arrow-rs](https://github.com/apache/arrow-rs/pull/4260), which makes me think 
that alignment is at least somewhat worked out?
   
   To me, a solution I'm thinking of is to get a large allocation, check the 
alignment of that and then hand out aligned chunks of that allocation. Since 
that would be upstream of the `object_store` crate (it produces the `Bytes` for 
the result), I'm asking here if there are any alignment guarantees of the 
`get_range(s)` function. If so, I'm happy and would only need to implement 
alignment-ness for other types.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to