Re: Re: Dynamic Columns

2015-01-21 Thread Jack Krupansky
Peter, At least from your description, the proposed use of the clustering column name seems at first blush to fully fit the bill. The point is not that the resulting clustered primary key is used to reference an object, but that a SELECT on the partition key references the entire object, which wil

Re: Is there a way to add a new node to a cluster but not sync old data?

2015-01-21 Thread Yatong Zhang
Yes, my cluster is almost full and there are lots of pending tasks. You helped me a lot and thank you Eric~ On Thu, Jan 22, 2015 at 11:59 AM, Eric Stevens wrote: > Yes, bootstrapping a new node will cause read loads on your existing nodes > - it is becoming the owner and replica of a whole new s

Re: Is there a way to add a new node to a cluster but not sync old data?

2015-01-21 Thread Eric Stevens
Yes, bootstrapping a new node will cause read loads on your existing nodes - it is becoming the owner and replica of a whole new set of existing data. To do that it needs to know what data it's now responsible for, and that's what bootstrapping is for. If you're at the point where bootstrapping a

Fwd: ReadTimeoutException in Cassandra 2.0.11

2015-01-21 Thread Neha Trivedi
Hello All, I am trying to process 200MB file. I am getting following Error. We are using (apache-cassandra-2.0.3.jar) com.datastax.driver.core. exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ONE (1 responses were required but only 0 replica responded) 1. Is it

Re: Is there a way to add a new node to a cluster but not sync old data?

2015-01-21 Thread Yatong Zhang
Thanks for the reply. The bootstrap of new node put a heavy burden on the whole cluster and I don't know why. So that' the issue I want to fix actually. On Mon, Jan 12, 2015 at 6:08 AM, Eric Stevens wrote: > Yes, but it won't do what I suspect you're hoping for. If you disable > auto_bootstrap

Re: get partition key from tombstone warnings?

2015-01-21 Thread Ian Rose
Ah, thanks for the pointer Philip. Is there any kind of formal way to "vote up" issues? I'm assuming that adding a comment of "+1" or the like is more likely to be *counter*productive. - Ian On Wed, Jan 21, 2015 at 5:02 PM, Philip Thompson < philip.thomp...@datastax.com> wrote: > There is an

Re: Re: Dynamic Columns

2015-01-21 Thread Peter Lin
I've written my fair share of crappy code, which became legacy. then I or someone else was left with supporting it and something newer. Isn't that the nature of software development. I forget who said this quote first, but I'm gonna borrow it "only pretty code is code that is in your head. once it

Re: Re: Dynamic Columns

2015-01-21 Thread Robert Coli
On Wed, Jan 21, 2015 at 2:09 PM, Peter Lin wrote: > on the topic of multiple incompatible API's I recommend you look at > SqlServer and Sybase. Most of the legacy RDBMS have multiple incompatible > API. Though in some cases, it is/was unavoidable. > My bet is that the small development team resp

Re: Re: Dynamic Columns

2015-01-21 Thread Peter Lin
everyone is different. I also recommend users take time to understanding every tool they use as much as time allows. We don't always have the luxury of time, but I see no point recommending laziness. I'm probably insane, since I also spend time reading papers on CRDT, paxos, query compilers, machi

Re: get partition key from tombstone warnings?

2015-01-21 Thread Philip Thompson
There is an open ticket for this improvement at https://issues.apache.org/jira/browse/CASSANDRA-8561 On Wed, Jan 21, 2015 at 4:55 PM, Ian Rose wrote: > When I see a warning like "Read 9 live and 5769 tombstoned cells in ... > " is there a way for me to see the partition key that this query was >

Re: Re: Dynamic Columns

2015-01-21 Thread Peter Lin
I apologize if I've offended you, but I clearly stated CQL3 supports dynamic columns. How it supports dynamic columns is different. If I'm reading you correctly, I believe we agree both thrift and CQL3 support dynamic columns. Where we differ that I feel the coverage for existing thrift use cases i

get partition key from tombstone warnings?

2015-01-21 Thread Ian Rose
When I see a warning like "Read 9 live and 5769 tombstoned cells in ... " is there a way for me to see the partition key that this query was operating on? The description in the original JIRA ticket ( https://issues.apache.org/jira/browse/CASSANDRA-6042) reads as though exposing this information w

Re: Re: Dynamic Columns

2015-01-21 Thread Robert Coli
On Wed, Jan 21, 2015 at 9:19 AM, Peter Lin wrote: > > I consistently recommend new users learn and understand both Thrift and > CQL. > FWIW, I consider this a disservice to new users. New users should use CQL, and not deploy against a deprecated-in-all-but-name API. Understanding non-CQL *storag

Re: Re: Dynamic Columns

2015-01-21 Thread Sylvain Lebresne
On Wed, Jan 21, 2015 at 6:19 PM, Peter Lin wrote: > the dynamic column can't be part of the primary key. The temporal entity > key can be the default UUID or the user can choose the field in their > object. Within our framework, we have concept of temporal links between one > or more temporal ent

Re: Compaction failing to trigger

2015-01-21 Thread Robert Coli
On Wed, Jan 21, 2015 at 10:10 AM, Flavien Charlon wrote: > https://engineering.eventbrite.com/what-version-of-cassandra-should-i-run/ >> >> > > This doesn't really answer my question, I asked whether this particular > bug (which I can't find in JIRA) is planned to be fixed in 2.1.3, not > whether

Re: How to know disk utilization by each row on a node

2015-01-21 Thread nitin padalia
Did you use cfstats and cfhistograms? On Jan 22, 2015 12:37 AM, "Edson Marquezani Filho" < edsonmarquez...@gmail.com> wrote: > Ok, nice tool, but I still can't see how much data each row occupies > on the SSTable (or am I missing something?). > > Obs: considering SSTables format, where rows are st

Re: How to know disk utilization by each row on a node

2015-01-21 Thread Edson Marquezani Filho
Ok, nice tool, but I still can't see how much data each row occupies on the SSTable (or am I missing something?). Obs: considering SSTables format, where rows are strictly sequential and sorted, a feature like that doesn't seem something very hard to implement, anyway. Wouldn't it be possible to c

Re: How do replica become out of sync

2015-01-21 Thread Flavien Charlon
Quite a few, see here: http://pastebin.com/SMnprHdp. In total about 3,000 ranges across the 3 nodes. This is with vnodes disabled. It was at least an order of magnitude worse when we had it enabled. Flavien On 20 January 2015 at 22:22, Robert Coli wrote: > On Mon, Jan 19, 2015 at 5:44 PM, Flav

Re: Re: Dynamic Columns

2015-01-21 Thread Peter Lin
the example you provided does not work for for my use case. CREATE TABLE t ( key blob, static my_static_column_1 int, static my_static_column_2 float, static my_static_column_3 blob, , dynamic_column_name blob, dynamic_column_value blob, PRIMARY KEY (key, dyna

Re: Compaction failing to trigger

2015-01-21 Thread Flavien Charlon
> > What version of Cassandra are you running? 2.1.2 Are they all "live"? Are there pending compactions, or exceptions regarding > compactions in your logs? Yes they are all live according to cfstats. There is no pending compaction or exception in the logs. https://engineering.eventbrite.com/

Re: Re: Dynamic Columns

2015-01-21 Thread Sylvain Lebresne
On Wed, Jan 21, 2015 at 4:44 PM, Peter Lin wrote: > I don't remember other people's examples in detail due to my shitty > memory, so I'd rather not misquote. > Fair enough, but maybe you shouldn't use "people's examples you don't remenber" as argument then. Those examples might be wrong or outda

Re: Re: Dynamic Columns

2015-01-21 Thread Peter Lin
I don't remember other people's examples in detail due to my shitty memory, so I'd rather not misquote. In my case, I mix static and dynamic columns in a single column family with primitives and objects. The objects are temporal object graphs with a known type. Doing this type of stuff is basicall

Re: Re: Dynamic Columns

2015-01-21 Thread Sylvain Lebresne
> I've chatted with several long time users of Cassandra and there's things > CQL3 doesn't support. > Would you care to elaborate then? Maybe a simple example of something (or multiple things since you used plural) in thrift that cannot be supported in CQL? And please note that I'm *not* saying th

Re: keyspace not exists?

2015-01-21 Thread Jason Wee
Thanks Rob, we keep this in mind for our learning journey. Jason On Wed, Jan 21, 2015 at 6:45 AM, Robert Coli wrote: > On Sun, Jan 18, 2015 at 8:55 PM, Jason Wee wrote: > >> two nodes running cassandra 2.1.2 and one running cassandra 2.1.1 >> > > For the record, this is an unsupported persiste

cassandra-stress - confusing documentation

2015-01-21 Thread Tzach Livyatan
Hi all I'm using cassandra-stress directly from apache-cassandra-2.1.2/tools/bin The documentation I found http://www.datastax.com/documentation/cassandra/2.1/cassandra/tools/toolsCStress_t.html is either too old or too advance, but does not match what I use. In particular, I fail to use the -key

Re: Count of column values

2015-01-21 Thread Poonam Ligade
Hi , Sorry for the previous incomplete message. I am using where clause as follows: select count(*) from trends where data1='abc' ALLOW FILTERING; How can i store this count output to any other column. Can you help with any wayround. Thanks, Poonam. On Wed, Jan 21, 2015 at 7:46 PM, Poonam Ligad

Count of column values

2015-01-21 Thread Poonam Ligade
Hi, I am newbie to Cassandra. I have to find out top 10 recent trends in data I have schema as follows create table trends( day int, data1 text, data2 map, PRIMARY KEY (day, data1)) ; I have to take count of duplicate values in data1 so that i can find top 10 data1 trends. 1. I tried adding an

Re: Re: Dynamic Columns

2015-01-21 Thread Peter Lin
I've studied the source code and I don't believe that statement is true. I've chatted with several long time users of Cassandra and there's things CQL3 doesn't support. Like I've said before. Thrift and CQL3 compliment each other. I totally understand some committers don't want the overhead due to

Re: row cache hit is costlier for partiton with large rows

2015-01-21 Thread Sylvain Lebresne
The row cache saves partition data off-heap, which means that every cache hit require copying/deserializing the cached partition into the heap, and the more rows per partition you cache, the long it will take. Which is why it's currently not a good cache too much rows per partition (unless you know

Re: Re: Dynamic Columns

2015-01-21 Thread Sylvain Lebresne
On Wed, Jan 21, 2015 at 3:46 AM, Peter Lin wrote: > > I don't understand why people [...] pretend it supports 100% of the use > cases. > Have you consider the possibly that it's actually true and you're just wrong by lack of knowledge? -- Sylvain

Re: Re: Dynamic Columns

2015-01-21 Thread Jonathan Lacefield
Hello, Peter highlighted the tradeoff between Thrift and CQL3 nicely in this case, i.e. requiring a different design approach for this solution. Collections do not sound like a good fit for your current challenge, but is there a different way to design/solve your challenge using CQL techniques?

row cache hit is costlier for partiton with large rows

2015-01-21 Thread nitin padalia
Hi, With two different families when I do a read, row cache hit is almost 15x costlier with larger rows (1 rows per partition), in comparison to partition with only 100 rows. Difference in two column families is one is having 100 rows per partition another 1 rows per partition. Schema for

Re: Versioning in cassandra while indexing ?

2015-01-21 Thread Kai Wang
depending on your data model, static column night be useful. https://issues.apache.org/jira/plugins/servlet/mobile#issue/CASSANDRA-6561 On Jan 21, 2015 2:56 AM, "Pandian R" wrote: > Hi, > > I just wanted to know if there is any kind of versioning system in > cassandra while indexing new data(like

Re: Versioning in cassandra while indexing ?

2015-01-21 Thread Pandian R
Awesome. Thanks a lot Graham. Will use the clock timestamp for versioning :) On Wed, Jan 21, 2015 at 2:02 PM, graham sanderson wrote: > I believe you can use “USING TIMESTAMP XXX” with your inserts which will > set the actual cell write times to the timestamp you provide. Then at least > on read

Re: Versioning in cassandra while indexing ?

2015-01-21 Thread graham sanderson
I believe you can use “USING TIMESTAMP XXX” with your inserts which will set the actual cell write times to the timestamp you provide. Then at least on read you’ll get the “latest” value… you may or may not incur an actual write of the old data to disk, but either way it’ll get cleaned up for yo