Jackie-Jiang commented on PR #12945: URL: https://github.com/apache/pinot/pull/12945#issuecomment-2078329847
`DEFAULT_NUM_DOCS_PER_CHUNK` is the thing we want to avoid in V4, so using it to calculate the target chunk size seems weird to me. How about we add 2 configs here: - `maxTargetChunkSize`: upper bound of the target chunk size - `targetDocsPerChunk`: reduce the target chunk size when max length is small. We can make it 1000 by default to have the desired behavior. I can imagine people want to disable this and always go with the `maxTargetChunkSize` for scan intensive case to reduce decompression -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
