somandal opened a new pull request, #16791: URL: https://github.com/apache/pinot/pull/16791
This PR builds on top of https://github.com/apache/pinot/pull/16585 which didn't handle the dependencies for time boundary among related tables. Changes introduced in this PR: - _globalLock - ReadWriteLock for updates to global data structures used in the class such as `_excludedServers`, `_routableServers` and `_enabledServerInstanceMap`. These cannot be broken down to a per table / per table-group scope - _routingTableBuildLocks - Map of locks keyed on the raw table name (to capture dependencies between OFFLINE and REALTIME tables that are dependent on each other for time boundary calculations) to Object locks. This is mostly to protect access to `_routingEntryMap` and allow more parallelism to build the map when routing entries are built for different tables. - _routingTableBuildStartTimeMs - Map of tableNameWithType to the last `buildRouting` call to skip building if it took too long to wait for the lock The above should improve parallelism among some of the operations that are safe to perform in parallel (e.g. those that don't modify the global data structures of the class). The table level locks help improve parallelism when performing `_routingEntryMap` operations at the table level. Building the routing table for logical tables relies on `buildRouting` for each table configured in the table list. Only if the `buildRouting` call doesn't set the TimeBoundaryManager, does the logical table code try to create one for the OFFLINE tables. Thus it should be sufficient to group the table level locks on the raw table name cc @Jackie-Jiang @xiangfu0 @abhishekbafna -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
