Skysheepwang opened a new issue #4995: URL: https://github.com/apache/incubator-doris/issues/4995
Currently, all types of pages (including DATA_PAGE, INDEX_PAGE, DICT_PAGE, SHORT_KEY_PAGE) in segment_v2 are cached by StoragePageCache in one single memory space. In many query situations, we expect to reuse the index pages in cache, but which are probably replaced by the large amount of data pages due to the LRU algorithm. Even in some extreme cases, the high frequency of LRU replacement usage caused by data pages will bring great pressure to the system. A separated page cache could be used to solve this problem. Implementation: - Change the "_cache" member of StoragePageCache class into "_data_cache" (for DATA) and "_index_page" (for INDEX, DICT, SHORT_KEY) and Add an "_index_cache_ratio" member as for the ratio of index cache. - Add a configuration item "index_cache_ratio" to set the ratio. The range is: 0 <= ratio <= 1, 0/1 stands for only data/index pages are cached. - Add "page_type" as an argument when reading pages, so that the StoragePageCache could use it to allocate the cache. Advantages: - By separating data and index pages, the hit rate of index pages could be highly improved and by tuning the ratio the hit rate of data pages will not be much decreased. - By setting the config::index_cache_ratio (including 0 or 1), users can adapt the cache ratio to fit their query situations. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
