alamb commented on code in PR #25481:
URL: https://github.com/apache/datafusion/pull/25481#discussion_r4050499837
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -1227,6 +1227,36 @@ pub fn take_function_args<const N: usize, T>(
})
}
+/// Returns the number of values covered by an offset buffer.
+///
+/// Slicing a variable-length array narrows its offsets but retains its entire
+/// values buffer. Use this span to size output buffers: it counts child
elements
+/// for lists and bytes for strings/binary arrays, including values in null
rows.
+/// An empty array still has one offset and therefore a span of zero.
+#[inline]
+pub fn offset_span_len<O: ArrowNativeType>(offsets: &OffsetBuffer<O>) -> usize
{
+ offset_span(offsets).1
+}
+
+/// Returns the start and length of the values covered by an offset buffer.
+///
+/// The returned pair can be passed to [`Array::slice`] to select the visible
+/// child values of a sliced list or map. For strings/binary arrays, the start
+/// and length are measured in bytes. Values in null rows are included.
+/// An empty array has a length of zero but may have a nonzero start.
+///
+/// ```
+/// # use arrow::buffer::OffsetBuffer;
+/// # use datafusion_common::utils::offset_span;
+/// let offsets = OffsetBuffer::new(vec![100_i32, 103, 108].into());
+/// assert_eq!(offset_span(&offsets), (100, 8));
+/// ```
+#[inline]
+pub fn offset_span<O: ArrowNativeType>(offsets: &OffsetBuffer<O>) -> (usize,
usize) {
+ let start = offsets[0].as_usize();
+ (start, offsets.last().unwrap().as_usize() - start)
Review Comment:
FYI this will logically conflict with
https://github.com/apache/datafusion/pull/25335 where the signature for
first/last changed
##########
datafusion/common/src/utils/mod.rs:
##########
@@ -1244,15 +1274,10 @@ pub fn list_values(array: &dyn Array) ->
Result<ArrayRef> {
fn sliced_list_values<O: OffsetSizeTrait>(list: &GenericListArray<O>) ->
ArrayRef {
let values = list.values();
- let offsets = list.offsets();
-
- if let (Some(first), Some(last)) = (offsets.first(), offsets.last()) {
- let first = first.as_usize();
- let last = last.as_usize();
+ let (start, len) = offset_span(list.offsets());
Review Comment:
I think this is much easier to read -- thank you @neilconway
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
