cambyzju commented on code in PR #12293:
URL: https://github.com/apache/doris/pull/12293#discussion_r965870153
##########
be/src/olap/rowset/segment_v2/column_reader.cpp:
##########
@@ -410,18 +410,33 @@ Status ArrayFileColumnIterator::next_batch(size_t* n,
ColumnBlockView* dst, bool
ColumnBlock* array_block = dst->column_block();
auto* array_batch =
static_cast<ArrayColumnVectorBatch*>(array_block->vector_batch());
- // 1. read n offsets
+ // 1. read n+1 offsets
+ array_batch->offsets()->resize(*n + 1);
ColumnBlock offset_block(array_batch->offsets(), nullptr);
- ColumnBlockView offset_view(&offset_block,
- dst->current_offset() + 1); //
offset应该比collection的游标多1
+ ColumnBlockView offset_view(&offset_block);
bool offset_has_null = false;
- RETURN_IF_ERROR(_length_iterator->next_batch(n, &offset_view,
&offset_has_null));
+ RETURN_IF_ERROR(_offset_iterator->next_batch(n, &offset_view,
&offset_has_null));
DCHECK(!offset_has_null);
if (*n == 0) {
return Status::OK();
}
- array_batch->get_offset_by_length(dst->current_offset(), *n);
+
+ if (_offset_iterator->get_current_page()->has_remaining()) {
+ // peek read one more to get the last array's length
+ size_t i = 1;
+ bool offsets_has_null = false;
+ ordinal_t save = _offset_iterator->get_current_ordinal();
+ RETURN_IF_ERROR(_offset_iterator->next_batch(&i, &offset_view,
&offsets_has_null));
Review Comment:
should we use another function to replace next_batch, to avoid twice call of
next_batch?
for example, we may has a function:
next_batch_for_offsets() {
// after normal read, always append more offset at the end;
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
