somandal opened a new pull request, #15360: URL: https://github.com/apache/pinot/pull/15360
This PR enhances the needReload pre-check done as part of table rebalancer to only fetch the status from the list of servers currently assigned based on the current assignment (ideal state). Without setting this explicit list, the existing needReload API fetches the list of servers tagged based on the table's server tenant tag. Sometimes this results in failed responses from servers that are not assigned any segments (e.g. they are newly added and that's why we're trying to rebalance) and an ERROR status can be returned from the pre-check indicating that needReload must be run manually. While the current needReload API is enough for most cases, when it comes to rebalance, we often want to move segments to new servers (uplift, tenant migration) or off of existing servers (downlift). The tenant tags should have already been updated prior to trying to rebalance, so the server list fetched is not correct. Another issue is that when some servers are removed, the existing needReload API will not fetch the status from the servers going away (which were untagged). This may be less of a concern since these segments will be deleted anyways, but it may still be useful to know the correct status Testing done: - Added a test case to `OfflineClusterIntegrationTest` to catch this scenario. Without this change, the `testRebalancePreChecks()` test will fail. With this change, it passes - Tested manually using `HybridQuickStart` with 4 servers to simulate multiple scenarios of tagging / untagging servers - Ran all the table rebalance unit tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
