Jackie-Jiang commented on code in PR #11729:
URL: https://github.com/apache/pinot/pull/11729#discussion_r1353373261
##########
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##########
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId, T context) {
default int getNumValuesMV(int docId, T context) {
throw new UnsupportedOperationException();
}
+
+ /**
+ * This class represents the buffer byte ranges accessed while reading a
given docId.
+ * See {@link ValueRangeProvider} for more details.
+ */
+ @AllArgsConstructor
+ @EqualsAndHashCode
+ @Getter
+ class ValueRange {
+ private final long _offset;
+ private final int _sizeInBytes;
+
+ public static ValueRange newByteRange(long offset, int sizeInBytes) {
Review Comment:
Is this useful? This is the same as the constructor
##########
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##########
@@ -916,4 +916,83 @@ default byte[][] getBytesMV(int docId, T context) {
default int getNumValuesMV(int docId, T context) {
throw new UnsupportedOperationException();
}
+
+ class ValueRange {
+ private final long _offset;
+ private final int _size;
+ // To tell if size uses bit or byte as unit, for fwd index reader reading
values of fixed bits.
+ private final boolean _isSizeOfBit;
+
+ public static ValueRange newByteRange(long offset, int sizeInBytes) {
+ return new ValueRange(offset, sizeInBytes, false);
+ }
+
+ public static ValueRange newBitRange(long offset, int sizeInBits) {
+ return new ValueRange(offset, sizeInBits, true);
+ }
+
+ private ValueRange(long offset, int size, boolean isSizeOfBit) {
+ _offset = offset;
+ _size = size;
+ _isSizeOfBit = isSizeOfBit;
+ }
+
+ public long getOffset() {
+ return _offset;
+ }
+
+ public int getSize() {
+ return _size;
+ }
+
+ public boolean isSizeOfBit() {
+ return _isSizeOfBit;
+ }
+
+ @Override
+ public String toString() {
+ return "Range{" + "_offset=" + _offset + ", _size=" + _size + ",
_isSizeOfBit=" + _isSizeOfBit + '}';
+ }
+ }
+
+ /**
+ * An interface that enables the caller to get byte ranges accessed while
reading a given docId.
+ * This can be used to eventually allow prefetching of exact byte ranges
during query processing,
+ * if the docIds to be fetched are known beforehand.
+ *
+ * This interface will be implemented by all the forward index readers.
+ */
+ interface ValueRangeProvider<T extends ForwardIndexReaderContext> {
Review Comment:
I'd also suggest directly adding it into `ForwardIndexReader`. We may add
default implementation for the methods, and potentially add `boolean
isByteRangeSupported()`
##########
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##########
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId, T context) {
default int getNumValuesMV(int docId, T context) {
throw new UnsupportedOperationException();
}
+
+ /**
+ * This class represents the buffer byte ranges accessed while reading a
given docId.
+ * See {@link ValueRangeProvider} for more details.
+ */
+ @AllArgsConstructor
+ @EqualsAndHashCode
+ @Getter
+ class ValueRange {
Review Comment:
Suggest naming it `ByteRange`
##########
pinot-segment-spi/src/main/java/org/apache/pinot/segment/spi/index/reader/ForwardIndexReader.java:
##########
@@ -916,4 +919,67 @@ default byte[][] getBytesMV(int docId, T context) {
default int getNumValuesMV(int docId, T context) {
throw new UnsupportedOperationException();
}
+
+ /**
+ * This class represents the buffer byte ranges accessed while reading a
given docId.
+ * See {@link ValueRangeProvider} for more details.
+ */
+ @AllArgsConstructor
+ @EqualsAndHashCode
+ @Getter
+ class ValueRange {
+ private final long _offset;
+ private final int _sizeInBytes;
+
+ public static ValueRange newByteRange(long offset, int sizeInBytes) {
+ return new ValueRange(offset, sizeInBytes);
+ }
+
+ @Override
+ public String toString() {
+ return "Range{" + "_offset=" + _offset + ", _size=" + _sizeInBytes + '}';
+ }
+ }
+
+ /**
+ * An interface that enables the caller to get byte ranges accessed while
reading a given docId.
+ * This can be used to eventually allow prefetching of exact byte ranges
during query processing,
+ * if the docIds to be fetched are known beforehand.
+ *
+ * This interface will be implemented by all the forward index readers.
+ */
+ interface ValueRangeProvider<T extends ForwardIndexReaderContext> {
+ /**
+ * Returns a list of {@link ValueRange} that represents all the distinct
+ * buffer byte ranges (absolute offset, sizeInBytes) that are accessed
when reading the given (@param docId}
+ * @param docId to find the range for
+ * @param context Reader context
+ * @param ranges List of {@link ValueRange} to which the applicable value
ranges will be added
+ */
+ void recordDocIdByteRanges(int docId, T context, List<ValueRange> ranges);
Review Comment:
Will the caller use this method when the value is fixed length?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
