I usually advise against running Cassandra (or most databases) inside Kubernetes. It might look like it simplifies operations, but in my experience, it tends to introduce more complexity than it solves.
With Cassandra specifically, Kubernetes may reschedule pods for reasons outside your control (e.g., node pressure, restarts, upgrades). This can lead to topology violations — for example, all replicas ending up in the same physical rack, defeating the purpose of proper rack and datacenter awareness. Another major issue is storage. Cassandra expects fast, local disks close to the compute layer. While Kubernetes StatefulSets can use PersistentVolumes, these are often network-attached and may not offer the performance or locality guarantees Cassandra needs. And if your pods get rescheduled, depending on your storage class and cloud provider, you may run into delays or errors reattaching volumes. Using an operator like K8ssandra doesn't necessarily eliminate these problems — it just adds another tool to manage within the puzzle. Luciano Greiner On Thu, Jun 12, 2025 at 6:20 AM Dor Laor via user <user@cassandra.apache.org> wrote: > > It's possible to manage Cassandra well both with VMs and containers. > As you'd be running one container per VM, there is no significant advantage > for > containers. K8s provides nice tooling and some methodological enforcement > which > brings order to the setup but if the team aren't top notch experts in k8s, > it's not worth > the trouble and the limitations that come with it (networking outside the k8s > cluster, etc). > It's good to have fewer layers. Most users run databases outside of > containers. > > On Wed, Jun 11, 2025 at 11:36 PM Raymond Yu <rayyu...@gmail.com> wrote: >> >> Hi Cassandra community, >> >> I would like to ask for your expert opinions regarding a discussion we're >> having about deploying Cassandra on AWS EC2 vs. AWS ECS. For context, we >> have a small dedicated DB engineering team that is familiar with operating >> and supporting Cassandra on EC2 for many customer teams. However, one team >> has developed custom tooling for operating Cassandra on ECS (EC2-backed) and >> would like for us to migrate to it for their Cassandra needs, which has >> spawned this discussion (K8ssandra was considered, but that team did not >> want to use Kubernetes). >> >> Further context on our team and experience: >> - Small dedicated team supporting Cassandra (and other DBs) >> - Familiar with operating EC2 on Cassandra >> - Familiar with standard IaC tools and languages >> (Ansible/Terraform/Python/etc.) >> - Only deploy in AWS >> >> Discussed points regarding staying with EC2: >> - Existing team experience and automation in deploying Cassandra on EC2 >> - Simpler solution is easier to support and maintain >> - Almost all documentation we can find and use is specific to deploying on >> EC2 >> - Third party support is familiar with EC2 by default >> - Lower learning curve is lower for engineers to onboard >> - More hands-on maintenance regarding OS upgrades >> - Less modern solution >> >> Discussed points regarding using the new ECS solution: >> - Containers are the more modern solution >> - Node autoheal feature in addition to standard C* operations via a control >> plane >> - Higher tool and architecture complexity that requires ramp-up in order to >> use and support effectively >> - We're on our own for potential issues with the tool itself after it would >> be handed off >> - No demonstrated performance gain over EC2-based clusters >> - Third-party support would be less familiar with dealing with ECS issues >> - Deployed on EC2 under the hood (one container per VM), so the underlying >> architecture is the same between both solutions >> >> Given that context, our team generally feels that there is little marginal >> benefit given the cost of ramp up and supporting a custom tool, but there >> has also been a request for harder evidence and outside opinions on the >> topic. It has been hard to find documentation of this specific comparison on >> EC2 vs ECS to reference. We'd love to hear your thoughts on our context, but >> also are interested in any general recommendations for one over the other. >> Thanks in advance! >> >> Best, >> Raymond Yu
