In my mind , it may be better to support most cloud storage : aws,
azure,gcp,aliyun and so on . We may make it a plugable. But in that way, it
seems there may need a filesystem interface layer for object storage. And
should we support ,distributed system like hdfs ,or something else. We
should firs
The intention of the CEP is to lay the groundwork to allow development of
ChannelProxyFactories that are pluggable in Cassandra. In this way any
storage system can be a candidate for Cassandra storage provided
FileChannels can be created for the system.
As I stated before I think that there may b
> it may be better to support most cloud storage
> It simply only supports S3, which feels a bit customized for a certain user
> and is not universal enough.Am I right ?
I agree w/the eventual goal (and constraint on design now) of supporting most
popular cloud storage vendors, but if we have som
Yeah, there is so much things to do as cassandra (share-nothing) is
different from some other system like hbase , So I think we can break the
final goal into multiple steps. first is what Claude proposed. But I
suggest that this design can make the interface more scalable and we can
consider the im
Hi list,
there is CASSANDRA-18773 we want to merge to 4.0 up to trunk (hence it will be
in 5.0 (alpha2)) and I want to be sure we are all OK with that (especially for
that 5.0 alpha release).
The patch is significantly speeding up the compaction throughput for cases when
you have a lot of SSTa
Hi,
Support for multiple storage backends including remote storage backends is a
pretty high value piece of functionality. I am happy to see there is interest
in that.
I think that `ChannelProxyFactory` as an integration point is going to quickly
turn into a dead end as we get into really usin
I agree with Ariel, the more suitable insertion point is probably the JDK level
FileSystemProvider and FileSystem abstraction.
It might also be that we can reuse existing work here in some cases?
> On 26 Sep 2023, at 17:49, Ariel Weisberg wrote:
>
>
> Hi,
>
> Support for multiple storage ba
We (DataStax) have a FileSystemProvider for Astra we can provide.
Works with S3/GCS/Azure.
I'll ask someone on our end to make it accessible.
This would work by having a bucket prefix per node. But there are lots
of details needed to support things like out of bound compaction
(mentioned in CEP).
Would it be possible to make Jimfs integration production-ready then? I see we
are using it in the tests already.
It might be one of the reference implementations of this CEP. If there is a
type of workload / type of nodes with plenty of RAM but no disk, some kind of
compute nodes, it would jus
Thanks all for the feedback! The patch has 2 +1s on trunk and back ported to
5.0, making sure it’s stable now; I plan to merge early this week.
> On Sep 21, 2023, at 2:07 PM, Ekaterina Dimitrova
> wrote:
>
> +1 from me too. Moreover, this work has started as part of the test efforts
> and id
I spent a little (very little) time building an S3 implementation using an
Apache licensed S3 filesystem package. I have not yet tested it but if
anyone is interested it is at
https://github.com/Aiven-Labs/S3-Cassandra-ChannelProxy
In looking at some of the code I think the Cassandra File class n
11 matches
Mail list logo
