richardstartin opened a new pull request #8102: URL: https://github.com/apache/pinot/pull/8102
The dimension table level locking strategy employed in `TableDataManager` creates contention at the segment level, to the extent that adding a lookup to a query slows it down 20x in a user system, with over 75% of samples reported in waiting for the lock to perform a primary key lookup. This PR employs a copy on write approach to replace the reference table when segments are updated, so that the lookups can be done without locking. <img width="1549" alt="Screenshot 2022-02-01 at 16 21 26" src="https://user-images.githubusercontent.com/16439049/152007593-38f5880d-e6e1-4e4d-b247-6d6469598c48.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
![https://user-images.githubusercontent.com/16439049/152007593-38f5880d-e6e1-4e4d-b247-6d6469598c48.png"](https://user-images.githubusercontent.com/16439049/152007593-38f5880d-e6e1-4e4d-b247-6d6469598c48.png"){kind=link}