Re: OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-12 Thread Tyler Hobbs
Well, in this case I would say you probably need about 300MB of space in the heap, since that's what you've calculated. The APIs are designed to let you do what you think is best and they definitely won't stop you from shooting yourself in the foot. Counting a huge row, or trying to grab every ro

Re: OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-12 Thread Dave Martin
Thanks Tyler. I was unaware of counters. The use case for column counts is really from a operational perspective, to allow a sysadmin to do adhoc checks on columns to see if something has gone wrong in software outside of cassandra. I think running a cassandra-cli command such as count, which mak

Re: OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-11 Thread Tyler Hobbs
Cassandra has to deserialize all of the columns in the row for get_count(). So from Cassandra's perspective, it's almost as much work as getting the entire row, it just doesn't have to send everything back over the network. If you're frequently counting 8 million columns (or really, anything signi

OutOfMemory on count on cassandra 0.6.8 for large number of columns

2010-12-11 Thread Dave Martin
Hi there, I see the following: 1) Add 8,000,000 columns to a single row. Each column name is a UUID. 2) Use cassandra-cli to run count keyspace.cf['myGUID'] The following is reported in the logs: ERROR [DroppedMessagesLogger] 2010-12-12 18:17:36,046 CassandraDaemon.java (line 87) Uncaught exce