siddharthteotia commented on issue #5554: URL: https://github.com/apache/incubator-pinot/issues/5554#issuecomment-650048355
This would be a good addition. We can potentially build a storage hierarchy as DRAM -> SSD/HDD -> Remote store (Deep Store). - Segments are adaptively (based on the query pattern) brought onto the compute nodes (servers) and stored on local storage (SSD or HDD). - Furthermore, heavily queried (and based on size) segments can be cached completely in DRAM on the servers. This would be helpful for systems that don't have SSD and thus paging overhead for memory mapped segments is non-trivial as compared to SSD. Azure Kusto implements this hierarchy. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
