mcvsubbu commented on PR #14771: URL: https://github.com/apache/pinot/pull/14771#issuecomment-2623404879
> > The cathcup is allowed only for 30 seconds post which the segment is downloaded. Correct me if I am wrong. How is that blocking everything ? > > Yes, correct but these threads are still blocking the remaining threads right (30s condition is checked after indexing each message batch)? Do you see any disadvantage of current approach. It's better to do timed acquire on semaphore than the current approach to avoid extremely slow ingestion. Do you really need the catchup option? I was even considering disabling that altogether. It is OK if the faster segments build and upload new segments and the slower ones simply download the finalized segments. I think we should put the entire catchup logic under a config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
