Thanks , So I think a jira can be created now. And I'd be happy to provide some help with this as well if needed.
Henrik Ingo <henrik.i...@datastax.com> 于2023年9月28日周四 00:21写道： > It seems I was volunteered to rebase the Astra implementation of this > functionality (FileSystemProvider) onto Cassandra trunk. (And publish it, > of course) I'll try to get going today or tomorrow, so that this > discussion can then benefit from having that code available for inspection. > And potentially using it as a soluttion to this use case. > > On Tue, Sep 26, 2023 at 8:04 PM Jake Luciani <jak...@gmail.com> wrote: > >> We (DataStax) have a FileSystemProvider for Astra we can provide. >> Works with S3/GCS/Azure. >> >> I'll ask someone on our end to make it accessible. >> >> This would work by having a bucket prefix per node. But there are lots >> of details needed to support things like out of bound compaction >> (mentioned in CEP). >> >> Jake >> >> On Tue, Sep 26, 2023 at 12:56 PM Benedict <bened...@apache.org> wrote: >> > >> > I agree with Ariel, the more suitable insertion point is probably the >> JDK level FileSystemProvider and FileSystem abstraction. >> > >> > It might also be that we can reuse existing work here in some cases? >> > >> > On 26 Sep 2023, at 17:49, Ariel Weisberg <ar...@weisberg.ws> wrote: >> > >> > >> > Hi, >> > >> > Support for multiple storage backends including remote storage backends >> is a pretty high value piece of functionality. I am happy to see there is >> interest in that. >> > >> > I think that `ChannelProxyFactory` as an integration point is going to >> quickly turn into a dead end as we get into really using multiple storage >> backends. We need to be able to list files and really the full range of >> filesystem interactions that Java supports should work with any backend to >> make development, testing, and using existing code straightforward. >> > >> > It's a little more work to get C* to creates paths for alternate >> backends where appropriate, but that works is probably necessary even with >> `ChanelProxyFactory` and munging UNIX paths (vs supporting multiple >> Fileystems). There will probably also be backend specific behaviors that >> show up above the `ChannelProxy` layer that will depend on the backend. >> > >> > Ideally there would be some config to specify several backend >> filesystems and their individual configuration that can be used, as well as >> configuration and support for a "backend file router" for file creation >> (and opening) that can be used to route files to the backend most >> appropriate. >> > >> > Regards, >> > Ariel >> > >> > On Mon, Sep 25, 2023, at 2:48 AM, Claude Warren, Jr via dev wrote: >> > >> > I have just filed CEP-36 [1] to allow for keyspace/table storage >> outside of the standard storage space. >> > >> > There are two desires driving this change: >> > >> > The ability to temporarily move some keyspaces/tables to storage >> outside the normal directory tree to other disk so that compaction can >> occur in situations where there is not enough disk space for compaction and >> the processing to the moved data can not be suspended. >> > The ability to store infrequently used data on slower cheaper storage >> layers. >> > >> > I have a working POC implementation [2] though there are some issues >> still to be solved and much logging to be reduced. >> > >> > I look forward to productive discussions, >> > Claude >> > >> > [1] >> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations >> > [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory >> > >> > >> > >> >> >> -- >> http://twitter.com/tjake >> > > > -- > > Henrik Ingo > > c. +358 40 569 7354 > > w. www.datastax.com > > <https://www.facebook.com/datastax> <https://twitter.com/datastax> > <https://www.linkedin.com/company/datastax/> > <https://github.com/datastax/> > > -- you are the apple of my eye !
