Hudson build is back to normal : Cassandra-0.6 #38

[Apache Hudson Server]

See 

Re: Avro RPC?

[Eric Evans]

On Wed, 2010-12-22 at 11:00 -0600, Eric Evans wrote:
> So, Avro RPC.  Is anyone using this?  Is there anyone interested in
> seeing it maintained?
> 
> I'm concentrating on CQL[1][2], which for me will culminate in the
> creation of a new, application-specific transport, one that doesn't
> use either of the frameworks.  To me, the existing RPC framework is
> just something to piggy-back on until things are otherwise working,
> and I'm starting to think Thrift might be a better piggy here (read:
> it has more momentum).

There hasn't been very many people sounding off on this, but those that
have seem to be OK with calling it quits on the Avro interface.  Since I
brought this up during the holiday season, I'll give it another week
just in case someone who really cares has been offline.

To be clear though, I'm not really interested in pushing this forward
anymore, so it's not enough to simply want it, we need someone(s)
willing to step up. 

-- 
Eric Evans
eev...@rackspace.com

Re: Avro RPC?

[Edward Capriolo]

On Tue, Dec 28, 2010 at 11:30 AM, Eric Evans  wrote:
> On Wed, 2010-12-22 at 11:00 -0600, Eric Evans wrote:
>> So, Avro RPC.  Is anyone using this?  Is there anyone interested in
>> seeing it maintained?
>>
>> I'm concentrating on CQL[1][2], which for me will culminate in the
>> creation of a new, application-specific transport, one that doesn't
>> use either of the frameworks.  To me, the existing RPC framework is
>> just something to piggy-back on until things are otherwise working,
>> and I'm starting to think Thrift might be a better piggy here (read:
>> it has more momentum).
>
> There hasn't been very many people sounding off on this, but those that
> have seem to be OK with calling it quits on the Avro interface.  Since I
> brought this up during the holiday season, I'll give it another week
> just in case someone who really cares has been offline.
>
> To be clear though, I'm not really interested in pushing this forward
> anymore, so it's not enough to simply want it, we need someone(s)
> willing to step up.
>
> --
> Eric Evans
> eev...@rackspace.com
>
>


@Eric I agree with many of your sentiments.

The "avro summary" was/is somewhere between wishful thinking and
educated guesswork. In ~ May 2010 a shiny new Avro project went top
level apache status. Meanwhile thrift had no full time committers and
had some glaring bugs that had been open in thrift 0.4.0 (some around
php) that annoyed everyone.

However thrift did have a release 0.5.0. There are some projects that
use thrift, Hbase, Cassandra, and Hive. Thrift still delivers on
bindings for a number of languages.

Avro is in catchup mode to thrift. They are still evolving the
project, while still trying to add support for more languages. As far
as I can tell there is no flagship project build around Avro
end-to-end.

It would be a shame to see the Avro support go away from Cassandra
because of all the hard work that was put into it. However the
maintenance cost might outweigh it's benefits.

0.7RC1 local_quorum -> TimedOutException

[Thor Carpenter]

Hi All,

I am a bit stumped.  When reading at a CL of local_quorum I get 
TimedOutExceptions however when reading at any other CL, including ALL, 
everything works as expected.  Any write CL, including local_quorum, also works 
as expected.

I am running a 4 node 0.7 RC1 cluster in datacenter "DC1".  "DC2" is planned 
for the future but currently has no nodes.  I am using Hector 0.7.0-20.  

Here is my testing schema:

create keyspace GlobalCacheKeyspaceDC1RF4 with 
replication_factor = 4 and 
placement_strategy = 'org.apache.cassandra.locator.NetworkTopologyStrategy' 
and
strategy_options = [{DC1:4, DC2:0}];

use GlobalCacheKeyspaceDC1RF4;

create column family GlobalCacheCF with
comment = 'General purpose cache' and
column_type = 'Standard' and
comparator = 'BytesType' and
keys_cached = 20 and
rows_cached = 2000 and
gc_grace = 0 and
read_repair_chance = 0.0;


Here is my cassandra-topology.properties:

10.5.64.15=DC1:R1
10.5.64.28=DC1:R1
10.5.64.56=DC1:R1
10.5.64.39=DC1:R1

# default for unknown nodes
default=DC2:R1


Here is the exception I get when reading at a CL of local_quorum:

1:44:14.176 [main] ERROR m.p.c.connection.HConnectionManager - Could not 
fullfill request on this host CassandraClient
11:44:14.182 [main] ERROR m.p.c.connection.HConnectionManager - Exception: 
me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
at 
me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:27)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$15.execute(KeyspaceServiceImpl.java:598)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl$15.execute(KeyspaceServiceImpl.java:587)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:89)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:101)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:129)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.service.KeyspaceServiceImpl.getColumn(KeyspaceServiceImpl.java:604)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:57)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:52)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:65)
 [hector-0.7.0-20.jar:na]
at 
me.prettyprint.cassandra.model.thrift.ThriftColumnQuery.execute(ThriftColumnQuery.java:51)
 [hector-0.7.0-20.jar:na]


There are no warnings or errors in cassandra.log.

Any guidance would be appreciated.

Thanks,
-Thor

Re: Avro RPC?

[Ryan King]

On Tue, Dec 28, 2010 at 9:42 AM, Edward Capriolo  wrote:
> On Tue, Dec 28, 2010 at 11:30 AM, Eric Evans  wrote:
>> On Wed, 2010-12-22 at 11:00 -0600, Eric Evans wrote:
>>> So, Avro RPC.  Is anyone using this?  Is there anyone interested in
>>> seeing it maintained?
>>>
>>> I'm concentrating on CQL[1][2], which for me will culminate in the
>>> creation of a new, application-specific transport, one that doesn't
>>> use either of the frameworks.  To me, the existing RPC framework is
>>> just something to piggy-back on until things are otherwise working,
>>> and I'm starting to think Thrift might be a better piggy here (read:
>>> it has more momentum).
>>
>> There hasn't been very many people sounding off on this, but those that
>> have seem to be OK with calling it quits on the Avro interface.  Since I
>> brought this up during the holiday season, I'll give it another week
>> just in case someone who really cares has been offline.
>>
>> To be clear though, I'm not really interested in pushing this forward
>> anymore, so it's not enough to simply want it, we need someone(s)
>> willing to step up.
>>
>> --
>> Eric Evans
>> eev...@rackspace.com
>>
>>
>
>
> @Eric I agree with many of your sentiments.
>
> The "avro summary" was/is somewhere between wishful thinking and
> educated guesswork. In ~ May 2010 a shiny new Avro project went top
> level apache status. Meanwhile thrift had no full time committers and
> had some glaring bugs that had been open in thrift 0.4.0 (some around
> php) that annoyed everyone.
>
> However thrift did have a release 0.5.0. There are some projects that
> use thrift, Hbase, Cassandra, and Hive. Thrift still delivers on
> bindings for a number of languages.
>
> Avro is in catchup mode to thrift. They are still evolving the
> project, while still trying to add support for more languages. As far
> as I can tell there is no flagship project build around Avro
> end-to-end.
>
> It would be a shame to see the Avro support go away from Cassandra
> because of all the hard work that was put into it. However the
> maintenance cost might outweigh it's benefits.

Agreed. Thrift's progress has improved a lot since we first talked
about using Avro RPC. In the meantime Avro RPC progress has slowed
(the focus is on the Avro storage implementations).

I think it'd be fair to give up on Avro RPC for client rpc for now*.
It doesn't deliver enough over Thrift anymore.

-ryan

* I'm still a fan of using avro for data storage and anywhere else we
currently do custom serialization

Re: interest in minor contention issues?

[Jonathan Ellis]

Hi Jason,

Thanks for looking into this!

What tool are you using?  I belatedly noticed your tickets (we should
have them resolved for 0.6.9) and it might be useful to do another
check against the 0.7 code.

On Fri, Aug 6, 2010 at 3:25 PM, Jason Fager  wrote:
> Hi all,
>
> I'm doing some work with the Java Debugger Interface, and I'm using
> Cassandra as a test bed.  I've found a few minor contention spots
> (example: MessageDigest.getInstance() being called for every
> invocation of FBUtilities.hash()), but before I start possibly
> spamming your jira with them, I wanted to gauge the general interest
> in knowing these kinds of minor issues.  I'm happy to report them as
> they come up while I'm doing my work, but if you consider them too
> minor to worry about until they actually show up as impacting
> performance during profiling, I don't want to annoy anyone.  Let me
> know.
>
> - Jason
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Re: interest in creating a "cassandra-gossip" library?

[Ryan King]

On Wed, Dec 22, 2010 at 11:07 AM, matthew hawthorne
 wrote:
> hello,
>
> I'm starting a project at my day job to deploy a gossip protocol
> implementation.  part of my initial work is to evaluate existing
> implementations.
>
> being loosely familiar with Cassandra, I read
> http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
> over the related code a bit.
>
> is there interest in breaking out the gossip-related portions of
> Cassandra into a library that could be reused by other projects?  I
> work on a team that is ready and willing to contribute heavily.  we'd
> just need some guidance as to how to structure the Cassandra
> subcomponent(s) and properly integrate them with the builds, tests,
> etc.

I think breaking gossip out could be very valuable. Gossip is a
somewhat under-understood and under-tested part of Cassandra.
Isolating it could help both of those situations. Ideally we'd not
change the normal build procedure for cassandra, but add a new target
for building a gossip library.

> here are a few examples of functionality we're looking to add:
>
> 1) hierarchical state - our use case is cross data center gossip,
> where we don't want every node in the 2 clusters communicating, but do
> want a node from cluster1 to send a summary of the cluster's state to
> cluster2, and vice versa.  essentially I'm talking about rolling up
> the state of multiple nodes into a single "virtual" node

I'd be interested in seeing the analysis here. There are some
challenging problems with having only one host do something in a
gossip based system.

> 2) mutual authentication - nodes verifying the identity of other nodes
> before gossipping
>
> 3) encryption - encrypted traffic, especially for the cross data center case
>
> any opinions on this? thanks in advance for any feedback!

-ryan

Re: 0.7RC1 local_quorum -> TimedOutException

[Jonathan Ellis]

Turn on debug logging to see what nodes the request gets sent to, and
which ones replied.

[dev to BCC]

On Tue, Dec 28, 2010 at 1:59 PM, Thor Carpenter
 wrote:
> Hi All,
>
> I am a bit stumped.  When reading at a CL of local_quorum I get 
> TimedOutExceptions however when reading at any other CL, including ALL, 
> everything works as expected.  Any write CL, including local_quorum, also 
> works as expected.
>
> I am running a 4 node 0.7 RC1 cluster in datacenter "DC1".  "DC2" is planned 
> for the future but currently has no nodes.  I am using Hector 0.7.0-20.
>
> Here is my testing schema:
>
> create keyspace GlobalCacheKeyspaceDC1RF4 with
>    replication_factor = 4 and
>    placement_strategy = 
> 'org.apache.cassandra.locator.NetworkTopologyStrategy' and
>    strategy_options = [{DC1:4, DC2:0}];
>
> use GlobalCacheKeyspaceDC1RF4;
>
> create column family GlobalCacheCF with
>    comment = 'General purpose cache' and
>    column_type = 'Standard' and
>    comparator = 'BytesType' and
>    keys_cached = 20 and
>    rows_cached = 2000 and
>    gc_grace = 0 and
>    read_repair_chance = 0.0;
>
>
> Here is my cassandra-topology.properties:
>
> 10.5.64.15=DC1:R1
> 10.5.64.28=DC1:R1
> 10.5.64.56=DC1:R1
> 10.5.64.39=DC1:R1
>
> # default for unknown nodes
> default=DC2:R1
>
>
> Here is the exception I get when reading at a CL of local_quorum:
>
> 1:44:14.176 [main] ERROR m.p.c.connection.HConnectionManager - Could not 
> fullfill request on this host CassandraClient
> 11:44:14.182 [main] ERROR m.p.c.connection.HConnectionManager - Exception:
> me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
>        at 
> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:27)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.service.KeyspaceServiceImpl$15.execute(KeyspaceServiceImpl.java:598)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.service.KeyspaceServiceImpl$15.execute(KeyspaceServiceImpl.java:587)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:89)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:101)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:129)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getColumn(KeyspaceServiceImpl.java:604)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:57)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:52)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:65)
>  [hector-0.7.0-20.jar:na]
>        at 
> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery.execute(ThriftColumnQuery.java:51)
>  [hector-0.7.0-20.jar:na]
>
>
> There are no warnings or errors in cassandra.log.
>
> Any guidance would be appreciated.
>
> Thanks,
> -Thor
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Re: interest in minor contention issues?

[Jason Fager]

Hi Jonathan,

No problem, thanks for all the work you and everyone on the dev team
do on Cassandra.

I have a small bit of wrapper code for scripting the Java Debugger
Interface, which is up at https://github.com/jfager/jdiscript.  I also
have a short writeup on it at
http://jasonfager.com/888-introducing-jdiscript/.  In a nutshell, I
just ran Cassandra with the debugger set to fire  events on JDI's
MonitorContendedEnterEvent, and handle them by printing out
stacktraces.

If I get some time in the next day or so, I'll take another shot at
the 0.7 codebase.

Best,
Jason



On Tue, Dec 28, 2010 at 3:14 PM, Jonathan Ellis  wrote:
> Hi Jason,
>
> Thanks for looking into this!
>
> What tool are you using?  I belatedly noticed your tickets (we should
> have them resolved for 0.6.9) and it might be useful to do another
> check against the 0.7 code.
>
> On Fri, Aug 6, 2010 at 3:25 PM, Jason Fager  wrote:
>> Hi all,
>>
>> I'm doing some work with the Java Debugger Interface, and I'm using
>> Cassandra as a test bed.  I've found a few minor contention spots
>> (example: MessageDigest.getInstance() being called for every
>> invocation of FBUtilities.hash()), but before I start possibly
>> spamming your jira with them, I wanted to gauge the general interest
>> in knowing these kinds of minor issues.  I'm happy to report them as
>> they come up while I'm doing my work, but if you consider them too
>> minor to worry about until they actually show up as impacting
>> performance during profiling, I don't want to annoy anyone.  Let me
>> know.
>>
>> - Jason
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com
>

Re: interest in creating a "cassandra-gossip" library?

[Stephen Connolly]

Is there a specific reason to avoid modularizing the source folders if we
are modularizing the jars?

In my experience, one big source tree leads to unwanted module dependencies,
and can lead to circular module dependencies, which render the modulization
moot anyway... (note I am not saying to go to m_v__ as a build tool, just to
keep module source code in separate trees so that intra-module dependencies
can be controlled... Of course I believe that other build tool is good at
helping, but if you've ever seen one of peter reilly's ANT builds you'll
know it can be done easily and beautifully in ANT... Admittedly he is on the
ANT pmc ;-) )

- Stephen

---
Sent from my Android phone, so random spelling mistakes, random nonsense
words and other nonsense are a direct result of using swype to type on the
screen
On 28 Dec 2010 20:17, "Ryan King"  wrote:
> On Wed, Dec 22, 2010 at 11:07 AM, matthew hawthorne
>  wrote:
>> hello,
>>
>> I'm starting a project at my day job to deploy a gossip protocol
>> implementation.  part of my initial work is to evaluate existing
>> implementations.
>>
>> being loosely familiar with Cassandra, I read
>> http://wiki.apache.org/cassandra/ArchitectureGossip and have looked
>> over the related code a bit.
>>
>> is there interest in breaking out the gossip-related portions of
>> Cassandra into a library that could be reused by other projects?  I
>> work on a team that is ready and willing to contribute heavily.  we'd
>> just need some guidance as to how to structure the Cassandra
>> subcomponent(s) and properly integrate them with the builds, tests,
>> etc.
>
> I think breaking gossip out could be very valuable. Gossip is a
> somewhat under-understood and under-tested part of Cassandra.
> Isolating it could help both of those situations. Ideally we'd not
> change the normal build procedure for cassandra, but add a new target
> for building a gossip library.
>
>> here are a few examples of functionality we're looking to add:
>>
>> 1) hierarchical state - our use case is cross data center gossip,
>> where we don't want every node in the 2 clusters communicating, but do
>> want a node from cluster1 to send a summary of the cluster's state to
>> cluster2, and vice versa.  essentially I'm talking about rolling up
>> the state of multiple nodes into a single "virtual" node
>
> I'd be interested in seeing the analysis here. There are some
> challenging problems with having only one host do something in a
> gossip based system.
>
>> 2) mutual authentication - nodes verifying the identity of other nodes
>> before gossipping
>>
>> 3) encryption - encrypted traffic, especially for the cross data center
case
>>
>> any opinions on this? thanks in advance for any feedback!
>
> -ryan

Re: 0.7RC1 local_quorum -> TimedOutException

[Thor Carpenter]

OK, attached are the logs. 

At time 15:55 I performed a write with CL = ONE, a subsequent read with CL = 
ONE and received correct results.  Similar results occur with any read CL other 
than LOCAL_QUORUM.

At time 15:58 I performed a write with CL = ONE, a subsequent read with CL = 
LOCAL_QUORUM and received a TimedOutException.

System.log and cassandra.log are from server dlc05-sea to corresponds with the 
hector error message "Closing client CassandraClient".  I 
haven't found anything in the logs that indicates the problem but maybe others 
with more discerning eyes can.




Please let me know if I can provide any additional information.

Thanks,
-Thor

On Dec 28, 2010, at 12:21 PM, Jonathan Ellis wrote:

> Turn on debug logging to see what nodes the request gets sent to, and
> which ones replied.
> 
> [dev to BCC]
> 
> On Tue, Dec 28, 2010 at 1:59 PM, Thor Carpenter
>  wrote:
>> Hi All,
>> 
>> I am a bit stumped.  When reading at a CL of local_quorum I get 
>> TimedOutExceptions however when reading at any other CL, including ALL, 
>> everything works as expected.  Any write CL, including local_quorum, also 
>> works as expected.
>> 
>> I am running a 4 node 0.7 RC1 cluster in datacenter "DC1".  "DC2" is planned 
>> for the future but currently has no nodes.  I am using Hector 0.7.0-20.
>> 
>> Here is my testing schema:
>> 
>> create keyspace GlobalCacheKeyspaceDC1RF4 with
>>replication_factor = 4 and
>>placement_strategy = 
>> 'org.apache.cassandra.locator.NetworkTopologyStrategy' and
>>strategy_options = [{DC1:4, DC2:0}];
>> 
>> use GlobalCacheKeyspaceDC1RF4;
>> 
>> create column family GlobalCacheCF with
>>comment = 'General purpose cache' and
>>column_type = 'Standard' and
>>comparator = 'BytesType' and
>>keys_cached = 20 and
>>rows_cached = 2000 and
>>gc_grace = 0 and
>>read_repair_chance = 0.0;
>> 
>> 
>> Here is my cassandra-topology.properties:
>> 
>> 10.5.64.15=DC1:R1
>> 10.5.64.28=DC1:R1
>> 10.5.64.56=DC1:R1
>> 10.5.64.39=DC1:R1
>> 
>> # default for unknown nodes
>> default=DC2:R1
>> 
>> 
>> Here is the exception I get when reading at a CL of local_quorum:
>> 
>> 1:44:14.176 [main] ERROR m.p.c.connection.HConnectionManager - Could not 
>> fullfill request on this host CassandraClient
>> 11:44:14.182 [main] ERROR m.p.c.connection.HConnectionManager - Exception:
>> me.prettyprint.hector.api.exceptions.HTimedOutException: TimedOutException()
>>at 
>> me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:27)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$15.execute(KeyspaceServiceImpl.java:598)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl$15.execute(KeyspaceServiceImpl.java:587)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:89)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:101)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:129)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.service.KeyspaceServiceImpl.getColumn(KeyspaceServiceImpl.java:604)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:57)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery$1.doInKeyspace(ThriftColumnQuery.java:52)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:65)
>>  [hector-0.7.0-20.jar:na]
>>at 
>> me.prettyprint.cassandra.model.thrift.ThriftColumnQuery.execute(ThriftColumnQuery.java:51)
>>  [hector-0.7.0-20.jar:na]
>> 
>> 
>> There are no warnings or errors in cassandra.log.
>> 
>> Any guidance would be appreciated.
>> 
>> Thanks,
>> -Thor
>> 
>> 
> 
> 
> 
> -- 
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of Riptano, the source for professional Cassandra support
> http://riptano.com