mapshen commented on issue #7192: URL: https://github.com/apache/pinot/issues/7192#issuecomment-885879389
> IIUC the problem here is during segment commit, there's no consumption. Instead of having a time scheduled for segment commit, we can find a way to start consuming for the new segment while the current segment is being committed. It would be great! @mcvsubbu I did before but didn’t find it very helpful. I can try redoing it later. That said, how fast could it be after tuning it to the bone in your experience? Like Jackie put it earlier, > With different columns and indexes, the segment creation time could be very different. Normally it might take ~3 minutes to build the segment (of 5m rows). An ideal state for us would be a few seconds in the 99th percentile. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
