Re: Deleted snapshot files filling up /var/lib/cassandra

2015-03-20 Thread David Wahler
gt; before), if you still have issues with repair not letting go of snapshotted > files even with free disk space I would look to raise a ticket in Jira. > > On 17 March 2015 at 12:46, David Wahler wrote: >> >> On Mon, Mar 16, 2015 at 6:51 PM, Ben Bromhead wrote: >&g

Re: Deleted snapshot files filling up /var/lib/cassandra

2015-03-17 Thread David Wahler
On Mon, Mar 16, 2015 at 6:51 PM, Ben Bromhead wrote: > If you are running a sequential repair (or have previously run a sequential > repair that is still running) Cassandra will still have the file descriptors > open for files in the snapshot it is using for the repair operation. Yeah, that align

Re: Deleted snapshot files filling up /var/lib/cassandra

2015-03-16 Thread David Wahler
On Mon, Mar 16, 2015 at 6:12 PM, Ben Bromhead wrote: > Cassandra will by default snapshot your data directory on the following > events: > > TRUNCATE and DROP schema events > when you run nodetool repair > when you run nodetool snapshot > > Snapshots are just hardlinks to existing SSTables so the

Re: Deleted snapshot files filling up /var/lib/cassandra

2015-03-16 Thread David Wahler
On Mon, Mar 16, 2015 at 5:28 PM, Jan wrote: > David; > > all the packaged installations use the /var/lib/cassandra directory. > Could you check your yaml config files and see if you are using this default > directory for backups > > May want to change it to a location with more disk space. We're

Deleted snapshot files filling up /var/lib/cassandra

2015-03-16 Thread David Wahler
We have a 16-node, globally-distributed cluster. running Cassandra 2.0.12. We're using the Datastax packages on CentOS 6.5. Even though the total amount of data on each server is only a few hundred MB (as measured by both du and the "load" metric), we're seeing a problem where the disk usage is st