Re: What happens if there is a collision?

2010-10-21 Thread Chris Dean
Jérôme Verstrynge writes: > You are making my point (lol). No matter what an application writes, > it should re-read its owns write for determinism for a given timestamp > when other application instances are writing in the same 'table'. The timestamp is an ever increasing clock so I wouldn't exp

[Q] MapReduce behavior and Cassandra's scalability for petabytes of data

2010-10-21 Thread Takayuki Tsunakawa
Hello, I'm evaluating whether Cassandra fits a certain customer well. The customer will collect petabytes of logs and analyze them. Could you tell me if my understanding is correct and/or give me your opinions? I'm sorry that the analysis requirement is not clear yet. 1. MapReduce behavior I read

Re: What happens if there is a collision?

2010-10-21 Thread Jérôme Verstrynge
On 22/10/2010 2:27, Nicholas Knight wrote: On Oct 22, 2010, at 7:41 AM, Jérôme Verstrynge wrote: Let's imagine that A initiates its column write at: 334450 ms with 'AAA' and timestamp 334450 ms Let's imagine that E initiates its column write at: 334451 ms with 'ZZZ'and timestamp 334450 ms (E i

Re: What happens if there is a collision?

2010-10-21 Thread Nicholas Knight
On Oct 22, 2010, at 7:41 AM, Jérôme Verstrynge wrote: > Let's imagine that A initiates its column write at: 334450 ms with 'AAA' and > timestamp 334450 ms > Let's imagine that E initiates its column write at: 334451 ms with 'ZZZ'and > timestamp 334450 ms > (E is the latest write) > > Let's ima

Re: What happens if there is a collision?

2010-10-21 Thread Jérôme Verstrynge
On 21/10/2010 23:40, Peter Schuller wrote: OK. Thanks for your answer. From an email exchange I had with Jonathan, all this means that one should re-read its writes with quorum to make sure they have not been overriden by timestamp-tie conflicts. I suggested to send feedback to writting node (in

Re: What happens if there is a collision?

2010-10-21 Thread Peter Schuller
> OK. Thanks for your answer. From an email exchange I had with Jonathan, all > this means that one should re-read its writes with quorum to make sure they > have not been overriden by timestamp-tie conflicts. I suggested to send > feedback to writting node (in the ACK) when such timestamps-tie con

Re: Commit log files

2010-10-21 Thread Aaron Morton
The commit log contains serialised RowMutation's which identify the CF by their internal ID. This identifies the Keysapce+CF pair. These are the same structures that are used when the mutation is applied during the initial request.   AaronOn 22 Oct, 2010,at 07:28 AM, Bill Hastings wrote:So how do

Re: error: identifier ONE is unqualified!

2010-10-21 Thread Aaron Morton
Look for  lib/thrift-rX.jar in the source. is the svn revision to use. http://wiki.apache.org/cassandra/InstallThriftNot sure if all those steps still apply, but it's what I did last time I felt like feeling some angst. AaronOn 22 Oct, 2010,at 08:57 AM, J T wrote:What is the latest versio

error: identifier ONE is unqualified!

2010-10-21 Thread J T
What is the latest version of Thrift that cassandra-trunk is is supposed to work with ? I know Thrift 0.2.0 works, I'm using that on an existing cassandra 0.7 trunk install. I recently tried setting up another casandra node and just got the latest version of Thrift, which is now at 0.6.0 but afte

Re: What happens if there is a collision?

2010-10-21 Thread Jérôme Verstrynge
On 21/10/2010 20:03, Peter Schuller wrote: My question is: is node E notified that it lost the battle against A? If yes how? If not, then it means that, although writes are atomic, they would not be deterministic. Node E would have to verify that its write was successful... Quorom is not really

Re: Commit log files

2010-10-21 Thread Bill Hastings
So how do we figure which column families belong to which keyspace or is that immaterial? On Thu, Oct 21, 2010 at 10:38 AM, aaron morton wrote: > It's co-mingled like hippies on the last day of a festival > http://wiki.apache.org/cassandra/ArchitectureCommitLog > > Aaron > > On 22 Oct 2010, at 03

Re: Reading a keyrange when using RP

2010-10-21 Thread Tyler Hobbs
It's possible with a second row/column family. In another Column Family, create a row where all of the column names are the row keys you want to be able to get a range of. Pick the comparator type for the column family so that the columns will be sorted in the order you want. You then get a slic

Re: What happens if there is a collision?

2010-10-21 Thread Peter Schuller
> My question is: is node E notified that it lost the battle against A? If yes > how? > > If not, then it means that, although writes are atomic, they would not be > deterministic. Node E would have to verify that its write was successful... Quorom is not really a special case in that sense; it ju

Re: Reading a keyrange when using RP

2010-10-21 Thread Utku Can Topçu
The goal is actually getting the rows in the range of "start","end" The order is not important at all. But what I can see is, this does not seem to be possible at all using RP. Am I wrong? On Thu, Oct 21, 2010 at 6:03 PM, Tyler Hobbs wrote: > You can't get rows back in order with RP. > > You c

Re: creating and dropping columnfamilies as a usecase

2010-10-21 Thread aaron morton
AFAIK it's not really the purpose of the dynamic schema functions. You may run into problems such as the caches are per CF and the CF's have a high memory overhead (3 * mem table MB) so your memory usage will jump around. Cloud Kick gather a lot of metrics this may help http://wiki.apache.org/

Re: Commit log files

2010-10-21 Thread aaron morton
It's co-mingled like hippies on the last day of a festival http://wiki.apache.org/cassandra/ArchitectureCommitLog Aaron On 22 Oct 2010, at 03:40, Bill Hastings wrote: > Does Cassandra maintain a commit log per table? Or are they co-mingled? > > -- > Cheers > Bill

Re: Reading a keyrange when using RP

2010-10-21 Thread Tyler Hobbs
You can't get rows back in order with RP. You can start out with a start key and end key of '' (empty) and use the row count argument instead, if your goal is paging the rows. To get the next page, start from the last key you got in the previous page. On Thu, Oct 21, 2010 at 8:44 AM, Utku Can To

creating and dropping columnfamilies as a usecase

2010-10-21 Thread Utku Can Topçu
Hi All, In the current project I'm working on. I have use case for hourly analyzing the rows. Since the 0.7x branch supports creating and dropping columnfamilies on the fly; My use case proposal will be: * Create a CF at the very beginning of every hour * At the end of the 1-hour period, analyze

Commit log files

2010-10-21 Thread Bill Hastings
Does Cassandra maintain a commit log per table? Or are they co-mingled? -- Cheers Bill

Re: Does Cassandra fit my requirements?

2010-10-21 Thread Craig Ching
Hi Peter, thanks for the response! I appreciate your validation of our needs. Sorry it's taken me so long to get back, I've been knocked on my ass with a nasty cold since late last week. On Sat, Oct 16, 2010 at 7:37 AM, Peter Schuller wrote: > > 2. What we really want for small users is just

Reading a keyrange when using RP

2010-10-21 Thread Utku Can Topçu
If I'm not mistaken cassandra has been providing support for keyrange queries also on RP. However when I try to define a keyrange such as, start: (key100, end: key200) I get an error like: InvalidRequestException(why:start key's md5 sorts after end key's md5. this is not allowed; you probably sho

Re: Warn while bootstrap

2010-10-21 Thread ruslan usifov
Thanks for replay How did you start the bootstrap and how are you checking if it has > completed? > > I start new node with true conf parameter, and than check ring state with follow command: freebsd# nodetool -h 192.168.0.37 -p 8080 ring, if bootstrap this command must give my two nodes, but i

Re: Warn while bootstrap

2010-10-21 Thread Aaron Morton
How did you start the bootstrap and how are you checking if it has completed? What do nodetool info and nodetool streams say? The dropped message log usually indicates the node is under load. The log messages normally include the thread and file and line number with them. That's makes it a bit

Re: Tried to add a new node but f*ed up

2010-10-21 Thread Aaron Morton
Check your tokens are correct using the algorithm discussed on this page http://wiki.apache.org/cassandra/Operations If all ok run a node tool cleanup to remove any data the nodes are no longer responsible for. (see section on bootstrapping in the link above) Finally, the load *may* include tom

Warn while bootstrap

2010-10-21 Thread ruslan usifov
Hello I have a little problem. In first i have one node, then i try to add second node, and bootstrap phase never ends. I use cassandra 0.66. All streams which sends data between nodes ends with successful, but after that on the first node(which was originally) i have follow warning: INFO 19:20