itschrispeck commented on code in PR #12945:
URL: https://github.com/apache/pinot/pull/12945#discussion_r1568163690
##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/fwd/SingleValueVarByteRawIndexCreator.java:
##########
@@ -38,6 +38,7 @@
*/
public class SingleValueVarByteRawIndexCreator implements ForwardIndexCreator {
private static final int DEFAULT_NUM_DOCS_PER_CHUNK = 1000;
+ private static final int TARGET_MIN_CHUNK_SIZE = 4 * 1024;
Review Comment:
This lower bound is debatable. 4KB is what we tested with and errs on the
side of minimal memory usage, but since it's uncompressed target size the
compressed chunk could be below disk read ahead value for many systems
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
