itschrispeck commented on code in PR #15234:
URL: https://github.com/apache/pinot/pull/15234#discussion_r1999345423
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeSegmentDataManager.java:
##########
@@ -868,6 +869,13 @@ public void run() {
// We could not build the segment. Go into error state.
_state = State.ERROR;
_segmentLogger.error("Could not build segment for {}",
_segmentNameStr);
+ if (_segmentCannotBuild) {
Review Comment:
+1 to putting behind a config. E.g. current behavior w/ json index using
default settings is to fail for malformed input, iiuc this would continuously
retry build up to the message? so we'd have many increasingly small segments as
we keep reaching the msg and cutting current num rows in half
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
