itschrispeck commented on code in PR #11739:
URL: https://github.com/apache/pinot/pull/11739#discussion_r1353978315
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/json/ImmutableJsonIndexReader.java:
##########
@@ -76,6 +80,67 @@ public ImmutableJsonIndexReader(PinotDataBuffer dataBuffer,
int numDocs) {
_docIdMapping = dataBuffer.view(invertedIndexEndOffset,
docIdMappingEndOffset, ByteOrder.LITTLE_ENDIAN);
}
+ @Override
+ public String[] getValuesForKeyAndDocs(String key, int[] docIds, Map<Object,
RoaringBitmap> cache) {
+ RoaringBitmap docIdMask = RoaringBitmap.bitmapOf(docIds);
+ int[] dictIds = getDictIdsForKey(key);
+ Map<Integer, String> docIdToValues = new HashMap<>(docIds.length);
+
+ if (cache.isEmpty()) {
+ for (int dictId = dictIds[0]; dictId < dictIds[1]; dictId++) {
+ // get docIds from posting list, convert these to the actual docIds
+ ImmutableRoaringBitmap flattenedDocIds =
_invertedIndex.getDocIds(dictId);
+ PeekableIntIterator it = flattenedDocIds.getIntIterator();
+ MutableRoaringBitmap realDocIds = new MutableRoaringBitmap();
+ while (it.hasNext()) {
+ realDocIds.add(getDocId(it.next()));
+ }
+ cache.put(dictId, realDocIds.toRoaringBitmap());
+ }
+ }
+
+ for (int dictId = dictIds[0]; dictId < dictIds[1]; dictId++) {
+ RoaringBitmap intersection = RoaringBitmap.and(cache.get(dictId),
docIdMask);
+ if (intersection.isEmpty()) {
+ continue;
+ }
+ // dictionary value lookup, stripping the path prefix
+ String val = _dictionary.getStringValue(dictId).substring(key.length() +
1);
+ for (int docId : intersection) {
+ docIdToValues.put(docId, val);
+ }
+ }
+
+ String[] values = new String[docIds.length];
+ for (int i = 0; i < docIds.length; i++) {
+ values[i] = docIdToValues.get(docIds[i]);
Review Comment:
If I iterate over the map I don't know which index of `values[]` corresponds
to the `docId` of the current Map.Entry, so I'd need to maintain another
mapping from `docId` to the index of the input `docIds[]` array right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
