raghavyadav01 commented on code in PR #17059:
URL: https://github.com/apache/pinot/pull/17059#discussion_r2457740223
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/store/SingleFileIndexDirectory.java:
##########
@@ -261,11 +263,29 @@ private void loadMap()
private void mapBufferEntries()
throws IOException {
+ // Phase 1: Prepare data structures - sort entries by start offset
+ // Use list to handle multiple entries with same start offset
+ List<IndexEntry> sortedEntries = _columnEntries.values().stream()
+ .sorted((e1, e2) -> Long.compare(e1._startOffset, e2._startOffset))
+ .collect(java.util.stream.Collectors.toList());
+
+ // Phase 2: Create buffers - handle all entries in sequential order
+ if (!sortedEntries.isEmpty()) {
+ createBuffersSequentially(sortedEntries);
+ }
+ }
+
+ /**
+ * Creates buffers for all entries in sequential order, handling both
zero-size and regular entries
+ */
+ private void createBuffersSequentially(List<IndexEntry> sortedEntries)
+ throws IOException {
+ // Use the original approach with TreeMap for better memory management
SortedMap<Long, IndexEntry> indexStartMap = new TreeMap<>();
Review Comment:
Thanks for catching it. Fixed it. We want to add non 0 size entry into
indexStartMap and ignore index map entries with remote forward index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
