gortiz commented on code in PR #13303:
URL: https://github.com/apache/pinot/pull/13303#discussion_r1706758696
##########
pinot-common/src/main/java/org/apache/pinot/common/datablock/BaseDataBlock.java:
##########
@@ -109,92 +114,27 @@ public BaseDataBlock(int numRows, @Nullable DataSchema
dataSchema, String[] stri
_numColumns = dataSchema == null ? 0 : dataSchema.size();
_fixDataSize = 0;
_stringDictionary = stringDictionary;
- _fixedSizeDataBytes = fixedSizeDataBytes;
- _fixedSizeData = ByteBuffer.wrap(fixedSizeDataBytes);
- _variableSizeDataBytes = variableSizeDataBytes;
- _variableSizeData = ByteBuffer.wrap(variableSizeDataBytes);
+ _fixedSizeData = PinotByteBuffer.wrap(ByteBuffer.wrap(fixedSizeDataBytes));
+ _variableSizeData =
PinotByteBuffer.wrap(ByteBuffer.wrap(variableSizeDataBytes));
_errCodeToExceptionMap = new HashMap<>();
}
- /**
- * Construct empty data table.
- */
- public BaseDataBlock() {
- _numRows = 0;
- _numColumns = 0;
+ public BaseDataBlock(int numRows, DataSchema dataSchema, String[]
stringDictionary,
+ DataBuffer fixedSizeData, DataBuffer variableSizeData) {
+ Preconditions.checkArgument(fixedSizeData.size() <= Integer.MAX_VALUE,
"Fixed size data too large ({} bytes",
+ fixedSizeData.size());
+ Preconditions.checkArgument(variableSizeData.size() <= Integer.MAX_VALUE,
"Variable size data too large ({} bytes",
+ variableSizeData.size());
+ _numRows = numRows;
+ _dataSchema = dataSchema;
+ _numColumns = dataSchema.size();
_fixDataSize = 0;
Review Comment:
That is not easy to do because the value is calculated in columnar and row
blocks in `computeBlockObjectConstants`. In order to calculate the value, this
method needs some attributes from the super class like the schema.
To be clear, the order is:
1. ColumnarDataBlock and RowDataBlock constructor call super
2. BaseDataBlock constructor initializes some internal attributes reading
the byte buffer
3. ColumnarDataBlock and RowDataBlock constructor use the internal
attributes to calculate `_fixDataSize`.
What we can do is to remove the field in BaseDataBlock and call an abstract
method. I'm going to modify the code to do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
