Jackie-Jiang commented on code in PR #10990:
URL: https://github.com/apache/pinot/pull/10990#discussion_r1271573773
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -619,4 +645,44 @@ public void close()
creators.addAll(_dictionaryCreatorMap.values());
FileUtils.close(creators);
}
+
+ /**
+ * Returns the original string if its length is within the allowed limit.
+ * If the string's length exceeds the limit,
+ * it returns a truncated version of the string with maintaining min or max
value.
+ *
+ */
+ @VisibleForTesting
+ static String getValueWithinLengthLimit(String value, boolean isMax,
DataType dataType) {
+ int length = value.length();
+
+ // if length is less, no need of trimming the value.
+ if (length <= METADATA_PROPERTY_LENGTH_LIMIT) {
+ return value;
+ }
+
+ String alteredValue;
+ // For Numeric Data Type(INT, LONG, DOUBLE, FLOAT) value longer than
METADATA_PROPERTY_LENGTH_LIMIT is not possible.
+ switch (dataType) {
+ case STRING:
+ case JSON:
+ if (isMax) {
+ alteredValue = value.substring(0, METADATA_PROPERTY_LENGTH_LIMIT - 1)
+ + (char) (value.charAt(METADATA_PROPERTY_LENGTH_LIMIT - 1) + 1);
+ } else {
+ alteredValue = value.substring(0, METADATA_PROPERTY_LENGTH_LIMIT);
+ }
+ break;
+ case BYTES:
+ byte[] shortByteValue = Arrays.copyOf(BytesUtils.toBytes(value),
(METADATA_PROPERTY_LENGTH_LIMIT / 2));
+ if (isMax) {
+ shortByteValue[METADATA_PROPERTY_LENGTH_LIMIT / 2 - 1] += 1;
Review Comment:
It won't cause an error, but will cause wrong behavior because it is no
longer larger than the original value.
The byte comparison is performed on the unsigned value, so we need to
prevent doing `+1` to `0xFF` which will result in `0`. We can skip all `0xFF`
until we find a value that is smaller, then replace it with `0xFF`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
