Re: Rows missing after new node bootstrapped

2010-11-17 Thread Christian Decker
after 3 days it's still not done migrating to the new nodes). Could this actually be the case? Regards, Chris B.T.W.: M/R and indices might mix well if we can just fetch the size of the index, and then we could create the splits telling them to "fetch from index starting from col n an

Re: Rows missing after new node bootstrapped

2010-11-15 Thread Christian Decker
I'm using tag cassandra-0.7.0-beta3. I wouldn't know why I need range scans since I perform a multi_get on the indexed keys. Regards, Chris On Sun, Nov 14, 2010 at 9:51 AM, Jonathan Ellis wrote: > Are you using a version with working range scans? > > On Sat, Nov 13, 2010 at

Rows missing after new node bootstrapped

2010-11-13 Thread Christian Decker
Hi all, I'm having some doubts about the current state of my cluster. I started with one node, filled it with some 10 million rows, then flushed and compacted the node. Then I ran a small pig script that read an index and fetched the matching rows, no problem until this point. Now I add a new node

Key to Token clarification

2010-11-05 Thread Christian Decker
Hi all, I'm trying to figure out some minor understanding problems. As I see it for each Keyspace each node takes care about a certain tokenRange (describe_ring gives me the assignment), these TokenRanges have a list of nodes that hold replicas, a start token and an end token. The tokens are of ty

describe_splits in 0.7

2010-11-02 Thread Christian Decker
Hi all, ever since I updated my development cluster I'm unable to use describe_splits to get information about the cluster to do a bulk load into hadoop. All I get is the following error: ERROR 15:44:24,498 Error occurred during processing of message. > java.lang.NullPointerException > at >

Fast way to find responsible nodes for a key?

2010-10-29 Thread Christian Decker
Hi all, I'm trying to find the most efficient way to find a node in the cluster (from the client side) that may hold a copy of rows I'm querying for. The scenario is quite simple: I have a hadoop job which reads an index and then has several thousands of keys, now I want to find a way to efficient

KeyRange over Long keys

2010-10-22 Thread Christian Decker
Ever since I started implementing my second level caches I've been wondering on how to deal with this, and thus far I've not found a good solution. I have a CF acting as a secondary index, and I want to make range queries against it. Since my keys are Long I simply went ahead and wrote them as the

Re: Creating and using indices

2010-10-08 Thread Christian Decker
On Thu, Oct 7, 2010 at 6:25 PM, Jonathan Ellis wrote: > On Thu, Oct 7, 2010 at 10:13 AM, Christian Decker > wrote: > > So basically my indices should work? Is there a simple way to check that, > so > > that we can exclude that? > > > > Are LTE working (or on

Re: Creating and using indices

2010-10-08 Thread Christian Decker
right? > > That's not yet supported, but should be soon. > > - Tyler > > > On Thu, Oct 7, 2010 at 10:13 AM, Christian Decker < > decker.christ...@gmail.com> wrote: > >> So basically my indices should work? Is there a simple way to check that, >> so

Re: Creating and using indices

2010-10-07 Thread Christian Decker
gt; right now is EQ, not LTE (or the others). >> >> >> On Thu, Oct 7, 2010 at 6:13 AM, Christian Decker < >> decker.christ...@gmail.com> wrote: >> >>> I'm currently trying to get started on secondary indices in Cassandra >>> 0.7.0svn, but

Creating and using indices

2010-10-07 Thread Christian Decker
I'm currently trying to get started on secondary indices in Cassandra 0.7.0svn, but without any luck so far. I have the following code that should create an index on ColA: KsDef ksDef = client.describe_keyspace("MyKeyspace"); > List cfs = ksDef.cf_defs; > String columnFamil

Re: LongType from user input

2010-09-30 Thread Christian Decker
through JMX, right? From there on I should be able to find my way around the system :-) On Thu, Sep 30, 2010 at 4:56 PM, Stu Hood wrote: > Take a look at the get_indexed_slices method in the 0.7.0-beta Thrift > interface. > > -Original Message----- > From: "Christian Decke

Re: LongType from user input

2010-09-30 Thread Christian Decker
BTW is there an official Roadmap which states when the 0.7 release is to be expected? On Thu, Sep 30, 2010 at 11:38 AM, Christian Decker < decker.christ...@gmail.com> wrote: > I just read through the tickets on Jira, and it appears that indices are > implemented in the 0.7 source

Re: LongType from user input

2010-09-30 Thread Christian Decker
n Thu, Sep 30, 2010 at 10:56 AM, Aaron Morton wrote: > AFAIK indexes are still in dev. The only example is in the system_tests.py > in the source tree. > > Aaron > > > On 30 Sep 2010, at 20:10, Christian Decker > wrote: > > Apparently I have blanked the 0.7 completely

Re: LongType from user input

2010-09-30 Thread Christian Decker
sandra? Is it using the > ColumnFamilyRecordReader? . The code in the internal RowIterator for that > class has an example calling the cluster to get to the comparators. > > Aaron > > > On 27 Sep, 2010,at 03:11 AM, Christian Decker > wrote: > > Hi Aaron, > > what chang

Re: LongType from user input

2010-09-26 Thread Christian Decker
representation to use them in column names and keys. Regards, Chris On Sun, Sep 26, 2010 at 5:20 AM, Aaron Morton wrote: > Things a changing in v0.7, the row keys are byte arrays. > > Not sure I understand your other concerns. > > Aaron > > > On 25 Sep 2010, at 08:10, C

Re: LongType from user input

2010-09-24 Thread Christian Decker
Tyler Hobbs wrote: > Yes, you can use describe_keyspace() and then look through the results. > It's a little ugly in 0.6, but it works. > > - Tyler > > > On Fri, Sep 24, 2010 at 11:25 AM, Christian Decker < > decker.christ...@gmail.com> wrote: > >> Well I&

Re: LongType from user input

2010-09-24 Thread Christian Decker
t. Could you clarify this? > > > On Fri, Sep 24, 2010 at 10:56 AM, Christian Decker < > decker.christ...@gmail.com> wrote: > >> Hi all, >> >> I'm having quite a dilemma with the CompareWith attribute. The Problem is >> that I have numeric ID

LongType from user input

2010-09-24 Thread Christian Decker
Hi all, I'm having quite a dilemma with the CompareWith attribute. The Problem is that I have numeric IDs that I'd like to use as row keys, only that I also have to offer a possibility to let users input them from std input. Since I cannot ask my users to input an 8byte sequence representing the I

Not overwriting values

2010-09-21 Thread Christian Decker
Hi all, I have a rather strange problem I'd like to address. As I understand it a write in cassandra always overwrites already existing data, so it is not possible to have a way to create an index pointing to the first entry matching some criteria. What I mean is that I have a CF which stores user

Join & Range Query performance

2010-08-28 Thread Christian Decker
I'm wondering what the performance considerations are on Join-like queries. I have a ColumnFamily that holds millions of records (not unusual as I understand) and I want to work on them using Pig and Hadoop. Until now we always fetched all rows in Cassandra and just filtered and worked on them. Th

Re: Pig + Cassandra = Connection errors

2010-08-21 Thread Christian Decker
running setup 1) it is still important to me to figure out what happens for the production system. Regards, Chris [1] http://snyke.net/tmp/screenshot_004.png -- Christian Decker Software Architect http://blog.snyke.net On Wed, Aug 18, 2010 at 2:17 PM, Christian Decker < decker.christ...@gmail

Re: Cassandra and Pig

2010-08-20 Thread Christian Decker
Regards, Chris -- Christian Decker Software Architect http://blog.snyke.net On Wed, Aug 18, 2010 at 8:09 PM, Stu Hood wrote: > Needing to manually copy the jars to all of the nodes would mean that you > aren't applying the Pig 'register ;' command properly. > > -----Origi

Re: Cassandra w/ Hadoop

2010-08-19 Thread Christian Decker
If, like me, you prefer to write your jobs on the fly try taking a look at Pig. Cassandra provides a loadfunc under contrib/pig/ in the source package which allows you to load data directly from Cassandra. -- Christian Decker Software Architect http://blog.snyke.net On Thu, Aug 19, 2010 at 7:23

Re: Pig + Cassandra = Connection errors

2010-08-19 Thread Christian Decker
/xeVkAkHu -- Christian Decker Software Architect http://blog.snyke.net

Re: Pig + Cassandra = Connection errors

2010-08-18 Thread Christian Decker
nodes? -- Christian Decker Software Architect http://blog.snyke.net On Wed, Aug 18, 2010 at 3:28 PM, Jonathan Ellis wrote: > why are you getting connection refused? do you have a firewall problem? > > On Wed, Aug 18, 2010 at 7:17 AM, Christian Decker > wrote: > > Hi all, &g

Re: Pig + Cassandra = Connection errors

2010-08-18 Thread Christian Decker
sec if you're getting timeouts. > > On Wed, Aug 18, 2010 at 8:17 AM, Christian Decker > wrote: > > Hi all, > > I'm trying to get Pig scripts to work on data in Cassandra and right now > I > > want to simply run the example-script.pig on a different Keyspace/CF

Pig + Cassandra = Connection errors

2010-08-18 Thread Christian Decker
$RowIterator.maybeInit(ColumnFamilyRecordReader.java:151) ... 13 more I checked that the cassandra cluster is running and all my 3 nodes are up and working. As far as I see it the Jobtracker retries when it get those errors but aborts once a large portion have failed. Any idea on why the Cluster keeps dropping connections or timing out? Regards, Chris -- Christian Decker Software Architect http://blog.snyke.net

Re: Cassandra and Pig

2010-08-18 Thread Christian Decker
r, or did I miss something? Regards, Chris -- Christian Decker Software Architect http://blog.snyke.net On Tue, Aug 17, 2010 at 4:04 PM, Christian Decker < decker.christ...@gmail.com> wrote: > Ok, by now it's getting very strange. I deleted the entire installation and > restarte

Re: Cassandra and Pig

2010-08-17 Thread Christian Decker
ed error: Error: java.lang.ClassNotFoundException: org.apache.thrift.TBase which is a bit different from my original error, but on the backend I get a classic ClassNotFoundException. Any ideas? -- Christian Decker Software Architect http://blog.snyke.net

Re: Cassandra and Pig

2010-08-15 Thread Christian Decker
rms some magic to automatically register Jars > containing classes directly referenced as UDFs. > > -Original Message- > From: "Christian Decker" > Sent: Friday, August 13, 2010 12:16pm > To: user@cassandra.apache.org > Subject: Re: Cassandra and Pig > > Wow, t

Re: Cassandra and Pig

2010-08-13 Thread Christian Decker
Original Message- > From: "Christian Decker" > Sent: Friday, August 13, 2010 11:30am > To: user@cassandra.apache.org > Subject: Cassandra and Pig > > Hi all, > > I'm trying to get Pig to read data from a Cassandra cluster, which I > thought > triv

Cassandra and Pig

2010-08-13 Thread Christian Decker
Hi all, I'm trying to get Pig to read data from a Cassandra cluster, which I thought trivial since Cassandra already provides me with the CassandraStorage class. Problem is that once I try executing a simple script like this: register /path/to/pig-0.7.0-core.jar;register /path/to/libthrift-r91713