- I think this is a great step forward. - Being able to move sstables around between tiers of storage is a feature Cassandra desperately needs, especially if one of those tiers is some sort of object storage - This looks like it's a foundational piece that enables that. Perhaps by a team that's already implemented this end to end? - Rather than building this piece by piece, I think it'd be awesome if someone drew up an end-to-end plan to implement tiered storage, so we can make sure we're discussing the whole final state, and not an implementation detail of one part of the final state?
On Sun, Sep 24, 2023 at 11:49 PM Claude Warren, Jr via dev < dev@cassandra.apache.org> wrote: > I have just filed CEP-36 [1] to allow for keyspace/table storage outside > of the standard storage space. > > There are two desires driving this change: > > 1. The ability to temporarily move some keyspaces/tables to storage > outside the normal directory tree to other disk so that compaction can > occur in situations where there is not enough disk space for compaction and > the processing to the moved data can not be suspended. > 2. The ability to store infrequently used data on slower cheaper > storage layers. > > I have a working POC implementation [2] though there are some issues still > to be solved and much logging to be reduced. > > I look forward to productive discussions, > Claude > > [1] > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations > [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory > > >
