Re: Query regarding SSTable timestamps and counts

2012-12-10 Thread B. Todd Burruss
my two cents ... i know this thread is a bit old, but the fact that odd-sized SSTABLEs (usually large ones) will hang around for a while can be very troublesome on disk space and planning. our data is temporal in cassandra, being deleted constantly. we have seen space usage in the 1+ TB range whe

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread aaron morton
>> upgradetables re-writes every sstable to have the same contents in the >> newest format. Agree. In the world of compaction, and excluding upgrades, have older sstables is expected. Cheers - Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thel

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread Ananth Gundabattula
Thanks a lot Aaron and Edward. The mail thread clarifies some things for me. For letting others know on this thread, running an upgradesstables did decrease our bloom filter false positive ratios a lot. ( upgradesstables was run not to upgrade from a casasndra version to a higher cassandra versio

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread Edward Capriolo
On Tue, Nov 20, 2012 at 5:23 PM, aaron morton wrote: > My understanding of the compaction process was that since data files keep > continuously merging we should not have data files with very old last > modified timestamps > > It is perfectly OK to have very old SSTables. > > But performing an upg

Re: Query regarding SSTable timestamps and counts

2012-11-20 Thread aaron morton
> My understanding of the compaction process was that since data files keep > continuously merging we should not have data files with very old last > modified timestamps It is perfectly OK to have very old SSTables. > But performing an upgradesstables did decrease the number of data files and

Re: Query regarding SSTable timestamps and counts

2012-11-19 Thread Rob Coli
On Sun, Nov 18, 2012 at 7:57 PM, Ananth Gundabattula wrote: > As per the above url, " After running a major compaction, automatic minor > compactions are no longer triggered, frequently requiring you to manually > run major compactions on a routine basis." ( Just before the heading Tuning > Column

Re: Query regarding SSTable timestamps and counts

2012-11-18 Thread Ananth Gundabattula
Hello Aaron, Thanks a lot for the reply. Looks like the documentation is confusing. Here is the link I am referring to: http://www.datastax.com/docs/1.1/operations/tuning#tuning-compaction > It does not disable compaction. As per the above url, " After running a major compaction, automatic min

Re: Query regarding SSTable timestamps and counts

2012-11-18 Thread aaron morton
> As per datastax documentation, a manual compaction forces the admin to start > compaction manually and disables the automated compaction (atleast for major > compactions but not minor compactions ) It does not disable compaction. it creates one big file, which will not be compacted until there