abhishekbafna opened a new pull request, #15881: URL: https://github.com/apache/pinot/pull/15881
This PR adds caching of the logical table config, logical table schema and reference offline and realtime tables for building logical table context. The cache is built using ZK data listener interface(s). The cache is comprised of individual maps of logical table config, schema config and table config. An additional map is used to maintain a mapping of table name to list of logical table names (actually a pair of table name and logical table name). This is used to ensure that table config is only removed it is not longer needed by any other logical table config. A table config can be part of multiple logical table configs. The implementation assumes that * A logical table schema can not be deleted before the logical table is deleted. * The reference offline and realtime tables from the logical table are cannot be deleted before the logical table is either modified or deleted. For the logical tables: * It listens to the new configs additions at the /LOGICAL/TABLE parent path and cache them. * The updates and delete to the logical table config are monitored at the individual path level. * The logical table config is parsed and relevant schema and table config are cached. Separate data change listener is added for each schema and table config path. * At the time of the deletion, configs are removed from the config and respective data change listener are also removed. For schema and table config: * Only data change listener is added which updates the config in place. * The delete operation are no-op as schema and table config delete can not happen being used in the logical table config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
