shounakmk219 commented on PR #13584: URL: https://github.com/apache/pinot/pull/13584#issuecomment-2274108292
Hey @mcvsubbu @sajjad-moradi , it is ok if the table storage spills over the quota by few segments before stopping the consumption and that said we can keep the quota validation + IS update logic at one place in segment validation manager. Also as per the suggestion of introducing `pauseStatus`, a single check on `pauseStatus` will cover both the pause reasons, eliminating the need of adding any extra checks during segment completion. > Can you state why you need the quota to be checked during segment completion? There is no strict storage quota imposition requirement, but also didn’t saw any major concern in putting the check during segment completion > remove latest segments (and adjust consumption offsets, of course) if the quota is exceeded We don’t want to do this as this will affect the query results (moving back in time). If it turns out that segment validation manager job frequency is not working for the storage quota imposition we can introduce a separate job when needed. Let me know if this sounds good. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
