Jackie-Jiang commented on code in PR #10990:
URL: https://github.com/apache/pinot/pull/10990#discussion_r1271267843
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -619,4 +645,44 @@ public void close()
creators.addAll(_dictionaryCreatorMap.values());
FileUtils.close(creators);
}
+
+ /**
+ * Returns the original string if its length is within the allowed limit.
+ * If the string's length exceeds the limit,
+ * it returns a truncated version of the string with maintaining min or max
value.
+ *
+ */
+ @VisibleForTesting
+ static String getValueWithinLengthLimit(String value, boolean isMax,
DataType dataType) {
+ int length = value.length();
+
+ // if length is less, no need of trimming the value.
+ if (length <= METADATA_PROPERTY_LENGTH_LIMIT) {
+ return value;
+ }
+
+ String alteredValue;
+ // For Numeric Data Type(INT, LONG, DOUBLE, FLOAT) value longer than
METADATA_PROPERTY_LENGTH_LIMIT is not possible.
+ switch (dataType) {
+ case STRING:
+ case JSON:
+ if (isMax) {
+ alteredValue = value.substring(0, METADATA_PROPERTY_LENGTH_LIMIT - 1)
+ + (char) (value.charAt(METADATA_PROPERTY_LENGTH_LIMIT - 1) + 1);
+ } else {
+ alteredValue = value.substring(0, METADATA_PROPERTY_LENGTH_LIMIT);
+ }
+ break;
+ case BYTES:
+ byte[] shortByteValue = Arrays.copyOf(BytesUtils.toBytes(value),
(METADATA_PROPERTY_LENGTH_LIMIT / 2));
+ if (isMax) {
+ shortByteValue[METADATA_PROPERTY_LENGTH_LIMIT / 2 - 1] += 1;
Review Comment:
This still can overflow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
