Re: Help on dynamic creation of CF

[aaron morton]

Moving to the User List 
Aaron

On 13 Oct 2010, at 18:44, gagandip Singh wrote:

> I am also new to the Cassandra world but I think that is not possible on 0.6
> version. This is feature is provided in 0.7 version which is in beta right
> now. You can download it from Cassandra site.
> 
> Thanks,
> Gagan
> 
> On Wed, Oct 13, 2010 at 11:05 AM, Wicked J  wrote:
> 
>> Hi,
>> I'm using Cassandra v0.6.4 and wondering how can my app. dynamically create
>> Column Families?
>> 
>> Thanks!
>> 

Help with creating a thrift call

[altanis]

I want to create a thrift call and would like to know how to go about it.

I think what I should do is this:

- add my call to cassandra.thrift (eg. string test(1:required string arg),
within 'service Cassandra')

- run thrift -gen java cassandra.thrift

- write a corresponding function in service/CassandraServer.java, (eg.:

public String test(String arg) throws TException
{
logger.debug("argument is "+arg);
return "Yay!";

}
)

- run ant on the whole project

Now I should be able to write a client that uses this command. Am I wrong?

I have tried all this, by copying the examples from ThriftExamples on the
wiki, but 1) they seem to be working with older Cassandra versions ( I am
using 0.7 beta1), 2) I seem to be doing something wrong as I get a
org.apache.thrift.transport.TTransportException when trying even
describe_cluster_name().

Any help will be appreciated. Please ask for any clarifications.

Alexander

Re: Help with creating a thrift call

[Gary Dusbabek]

On Wed, Oct 13, 2010 at 08:21,   wrote:
> I want to create a thrift call and would like to know how to go about it.
>
> I think what I should do is this:
>
> - add my call to cassandra.thrift (eg. string test(1:required string arg),
> within 'service Cassandra')
>
> - run thrift -gen java cassandra.thrift
>
> - write a corresponding function in service/CassandraServer.java, (eg.:
>
>    public String test(String arg) throws TException
>    {
>        logger.debug("argument is "+arg);
>        return "Yay!";
>
>    }
> )
>
> - run ant on the whole project
>
> Now I should be able to write a client that uses this command. Am I wrong?

No. It should just work.

>
> I have tried all this, by copying the examples from ThriftExamples on the
> wiki, but 1) they seem to be working with older Cassandra versions ( I am
> using 0.7 beta1), 2) I seem to be doing something wrong as I get a
> org.apache.thrift.transport.TTransportException when trying even
> describe_cluster_name().

Sounds like a framed/unframed mismatch.  Cassandra starts up in framed
mode, and your client needs to match.  When you set up your transport:
TTransport transport = new TFramedTransport(socket);

ThriftExamples on the wiki hasn't been updated to reflect this yet. :(

Gary.

>
> Any help will be appreciated. Please ask for any clarifications.
>
> Alexander
>

Build failed in Hudson: Cassandra #564

[Apache Hudson Server]

See 

Changes:

[jbellis] expose in-progress compaction type in jmx
patch by jbellis; reviewed by gdusbabek for CASSANDRA-1586

[jbellis] rename finishRemoval to forceRemoveCompletion.  patch by jbellis

[jbellis] cleanup parameters to handleStateRemoving.  patch by jbellis

[jbellis] add executor task queue size to CompactionManager pendingTasks.  
patch by jbellis

[brandonwilliams] Remove ConsistencyLevel.ZERO.  Patch by jbellis, reviewed by 
brandonwilliams for CASSANDRA-1607.

[jbellis] update comments to reflect new Stage.ANTIENTROPY name.  patch by 
jbellis

[jbellis] update listen_address, rpc_address comments

[jbellis] clean up, mostly removal of unnecessary Latch.  patch by jbellis

[jbellis] avoid blocking for dead nodes during removetoken.  patch by Nick 
Bailey; reviewed by jbellis for CASSANDRA-1605

--
[...truncated 1607 lines...]
[junit] Testsuite: org.apache.cassandra.io.BloomFilterTrackerTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.406 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.io.LazilyCompactedRowTest
[junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 2.846 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.io.sstable.LegacySSTableTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 3.631 sec
[junit] 
[junit] - Standard Error -
[junit]  WARN 14:08:01,692 Invalid file '.svn' in data directory 

[junit] -  ---
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.io.sstable.SSTableReaderTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 1.273 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.io.sstable.SSTableTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 6.552 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.io.sstable.SSTableWriterTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.762 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.io.util.BufferedRandomAccessFileTest
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.107 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.locator.DynamicEndpointSnitchTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.206 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.locator.NetworkTopologyStrategyTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.47 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: 
org.apache.cassandra.locator.OldNetworkTopologyStrategyTest
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.398 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: 
org.apache.cassandra.locator.ReplicationStrategyEndpointCacheTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.501 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.locator.SimpleStrategyTest
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.562 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.locator.TokenMetadataTest
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.478 sec
[junit] 
[junit] Cobertura: Loaded information on 965 classes.
[junit] Cobertura: Saved information on 965 classes.
[junit] Testsuite: org.apache.cassandra.service.AntiEntropyServiceTest
[junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 3.47 se

Re: 0.7 beta2 - Fatal exception from RowMutationVerbHandler

[Michael Moores]

I created CASSANDRA-1617

On Oct 12, 2010, at 1:51 PM, Michael Moores wrote:

> I have a cluster of 8 nodes with a replication factor of 3 and consistency of 
> QUORUM.
> When I stop one node in the cluster I end up with socket read timeouts to 
> other nodes:
> org.apache.thrift.transport.TTransportException: 
> java.net.SocketTimeoutException: Read timed out
> The timeout is set to 4 seconds.
> 
> The recent write latency on the other nodes becomes elevated, reaching 5-6 
> seconds.
> The load on the cluster is minimal.
> 
> I looked at the system log on an adjacent node in the ring and found this 
> output:
> 
> INFO [WRITE-kv2-app02.dev.real.com/172.27.109.32] 2010-10-12 12:13:36,712 
> OutboundTcpConnection.java (line 115) error writing to 
> kv2-app02.dev.real.com/172.27.109.32
> INFO [WRITE-kv2-app02.dev.real.com/172.27.109.32] 2010-10-12 12:13:50,336 
> OutboundTcpConnection.java (line 115) error writing to 
> kv2-app02.dev.real.com/172.27.109.32
> INFO [Timer-0] 2010-10-12 12:14:22,792 Gossiper.java (line 196) InetAddress 
> /172.27.109.32 is now dead.
> ERROR [MUTATION_STAGE:1315] 2010-10-12 12:14:24,917 
> DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor
> java.nio.BufferUnderflowException
>at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:127)
>at java.nio.ByteBuffer.get(ByteBuffer.java:675)
>at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:62)
>at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:50)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>at java.lang.Thread.run(Thread.java:619)
> ERROR [MUTATION_STAGE:1315] 2010-10-12 12:14:24,918 
> AbstractCassandraDaemon.java (line 88) Fatal exception in thread 
> Thread[MUTATION_STAGE:1315,5,main]
> java.nio.BufferUnderflowException
>at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:127)
>at java.nio.ByteBuffer.get(ByteBuffer.java:675)
>at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:62)
>at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:50)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>at java.lang.Thread.run(Thread.java:619)
> ERROR [MUTATION_STAGE:1605] 2010-10-12 12:14:28,919 
> DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor
> java.nio.BufferUnderflowException
>at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:127)
>at java.nio.ByteBuffer.get(ByteBuffer.java:675)
>at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:62)
>at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:50)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>at java.lang.Thread.run(Thread.java:619)
> 
> 
> 
> I restarted the previously stopped node, and the system recovers, but with a 
> few more underlflow exceptions:
> 
> INFO [GOSSIP_STAGE:1] 2010-10-12 12:15:44,537 Gossiper.java (line 594) Node 
> /172.27.109.32 has restarted, now UP again
> INFO [HINTED-HANDOFF-POOL:1] 2010-10-12 12:15:44,537 
> HintedHandOffManager.java (line 196) Started hinted handoff for endpoint 
> /172.27.109.32
> INFO [GOSSIP_STAGE:1] 2010-10-12 12:15:44,537 StorageService.java (line 643) 
> Node /172.27.109.32 state jump to normal
> INFO [HINTED-HANDOFF-POOL:1] 2010-10-12 12:15:44,538 
> HintedHandOffManager.java (line 252) Finished hinted handoff of 0 rows to 
> endpoint /172.27.109.32
> INFO [GOSSIP_STAGE:1] 2010-10-12 12:15:44,538 StorageService.java (line 650) 
> Will not change my token ownership to /172.27.109.32
> ERROR [MUTATION_STAGE:1635] 2010-10-12 12:15:45,083 
> DebuggableThreadPoolExecutor.java (line 103) Error in ThreadPoolExecutor
> java.nio.BufferUnderflowException
>at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:127)
>at java.nio.ByteBuffer.get(ByteBuffer.java:675)
>at 
> org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:62)
>at 
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:50)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>at java.lang.Thread.run(Thread.java:619)
> 
> 
> 

Re: Status of LT LTE without EQ op for 0.7 release

[Todd Nine]

Hi Tyler,
  That was the original scheme I was describing in the original email.
Unfortunately, I can have more that one value per column, so I actually
have to use super columns.  This way I can write more that one row key
for any given indexed value.  I'm concerned that this may not scale well
(at least on version 0.6).  However after looking at the limitations
page.

http://wiki.apache.org/cassandra/CassandraLimitations

It appears that the "row must fit in memory" has been removed.  I'll
move back to this scheme for my querying.

todd 
SENIOR SOFTWARE ENGINEER

todd nine| spidertracks ltd |  117a the square 
po box 5203 | palmerston north 4441 | new zealand 
P: +64 6 353 3395 | M: +64 210 255 8576  
E: t...@spidertracks.co.nz W: www.spidertracks.com 





On Tue, 2010-10-12 at 23:01 -0500, Tyler Hobbs wrote:

> I'm not completely sure I follow your scheme, but it's fairly to
> support
> GT, LT, etc with your own index.
> 
> Use a row for your index where the columns names are the data values
> you want to index.  If you set the comparator type (in your example,
> this
> would be LongType), you can perform a LT or GT query just by getting a
> slice of the index columns.  Store the original data row keys as the
> column
> values, and you're there.
> 
> - Tyler
> 
> 
> On Tue, Oct 12, 2010 at 9:33 PM, Todd Nine 
> wrote:
> 
> Thanks Johnathan,
> 
> A follow up question.  Will it be possible to migrate existing
> indexes
> in a future release as part of the upgrade path to support LT
> and LTE
> ops without equal?   In the meantime in my Datanucleus Plugin
> I was
> thinking I could do something like the following.  It's not
> efficient
> for space, but it will work and should hopefully be relatively
> efficient
> for querying.
> 
> 
> LT and LTE ops can be though of as the distance from the MAX
> value of
> any given data type.  For instance, if I had a data
> type :"ubershort",
> which goes from -200 to 200, I could say that an expression of
> <= 0 is
> really >= (distance) 200 from the maximum.  I could use this
> equation to
> calculate the "distance" to persist a distance value in a
> column named
> "_reverse". Which would effectively give me a reverse
> index.
> 
> 
> Then the value would simply be
> 
> storedValue = MAXVALUE-userVal.
> 
> >From there, whenever the user issues a < <= query, I would
> simply
> translate the value via the above equation and < becomes > and
> <=
> becomes >=.  Aside from the space issue of storage, do you see
> any other
> problems with this approach for a 0.7 compatible version of my
> plugin?
> 
> Thanks,
> Todd
> 
> 
> 
> 
> 
> 
> 
> On Wed, 2010-10-13 at 14:00 +1300, Todd Nine wrote:
> 
> > Fair enough!
> >
> >
> > Thanks Jonathan.
> >
> >
> > todd
> > SENIOR SOFTWARE ENGINEER
> >
> > todd nine| spidertracks ltd |  117a the square
> > po box 5203 | palmerston north 4441 | new zealand
> > P: +64 6 353 3395 | M: +64 210 255 8576
> > E: t...@spidertracks.co.nz W: www.spidertracks.com
> >
> >
> >
> >
> >
> >
> >
> > On Tue, 2010-10-12 at 18:47 -0500, Jonathan Ellis wrote:
> >
> > > On Tue, Oct 12, 2010 at 6:34 PM, Todd Nine
>  wrote:
> > > > Currently there is only indexing for LT and LTE
> expression when an EQ
> > > > operator is present.  Will it be possible to use the LT
> and LTE ops
> > > > without an EQ by the 0.7.0 release?
> > >
> > > No.
> > >
> > > >  If not, which of the following
> > > > would be more efficient?
> > > >
> > > > 1. Creating a dummy column of 1 byte that is indexed.
> > >
> > > This is basically the same as doing a full range scan,
> only less efficient.
> > >
> > > > 2. Use my previous indexing scheme of 2 Super CF for
> longs and strings
> > > > to get my < <= operations.  Where I use the following
> scheme.
> > >
> > > I'm not sure I follow but if it's better than doing a full
> range scan
> > > then it is better than 1. :)
> > >
> 
> 
>