ankitsultana commented on issue #13683: URL: https://github.com/apache/pinot/issues/13683#issuecomment-2251964454
Haven't thought deeply about this, but instead of adding a new routing config as in #13686, could we instead use a separate semaphore for blocking partial upsert table consumption in `PartitionConsumer#run` (or there might be some other better place)? The idea is that to ensure consistency for Partial Upserts, it's enough to make sure the consumption doesn't start before the previous segment is completely destroyed. To do that, using the current semaphore is a bit of an overkill since it straight up blocks the creation of the segment data manager. Instead, all we want to block in that case is the consumption, which could be done in a more fine-grained manner by doing it in `PartitionConsumer`. We won't need any new configs for this either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
