> are you really recommending I throw 4 years of work out and completely
rewrite code that works and has been tested?
Our codebase was about 3 years old, and we finished migrating it to CQL not
that long ago. It can definitely be frustrating to have to touch stable
code to modernize it. Our desi
> Where we differ that I feel the coverage for existing thrift use cases
isn't
> 100%. That may be right or wrong, but it is my impression.
Here's my problem: either CQL covers all existing thrift use cases or it
does
not (in which case the non supported use case should be pointed out). It's a
tec
@jack thanks for taking time to respond. I agree I could totally redesign
and rewrite it to fit in the newer CQL3 model, but are you really
recommending I throw 4 years of work out and completely rewrite code that
works and has been tested?
Ignoring the practical aspects for now and exploring the
Peter,
At least from your description, the proposed use of the clustering column
name seems at first blush to fully fit the bill. The point is not that the
resulting clustered primary key is used to reference an object, but that a
SELECT on the partition key references the entire object, which wil
I've written my fair share of crappy code, which became legacy. then I or
someone else was left with supporting it and something newer. Isn't that
the nature of software development.
I forget who said this quote first, but I'm gonna borrow it "only pretty
code is code that is in your head. once it
On Wed, Jan 21, 2015 at 2:09 PM, Peter Lin wrote:
> on the topic of multiple incompatible API's I recommend you look at
> SqlServer and Sybase. Most of the legacy RDBMS have multiple incompatible
> API. Though in some cases, it is/was unavoidable.
>
My bet is that the small development team resp
everyone is different. I also recommend users take time to understanding
every tool they use as much as time allows. We don't always have the luxury
of time, but I see no point recommending laziness.
I'm probably insane, since I also spend time reading papers on CRDT, paxos,
query compilers, machi
I apologize if I've offended you, but I clearly stated CQL3 supports
dynamic columns. How it supports dynamic columns is different. If I'm
reading you correctly, I believe we agree both thrift and CQL3 support
dynamic columns. Where we differ that I feel the coverage for existing
thrift use cases i
On Wed, Jan 21, 2015 at 9:19 AM, Peter Lin wrote:
>
> I consistently recommend new users learn and understand both Thrift and
> CQL.
>
FWIW, I consider this a disservice to new users. New users should use CQL,
and not deploy against a deprecated-in-all-but-name API. Understanding
non-CQL *storag
On Wed, Jan 21, 2015 at 6:19 PM, Peter Lin wrote:
> the dynamic column can't be part of the primary key. The temporal entity
> key can be the default UUID or the user can choose the field in their
> object. Within our framework, we have concept of temporal links between one
> or more temporal ent
the example you provided does not work for for my use case.
CREATE TABLE t (
key blob,
static my_static_column_1 int,
static my_static_column_2 float,
static my_static_column_3 blob,
,
dynamic_column_name blob,
dynamic_column_value blob,
PRIMARY KEY (key, dyna
On Wed, Jan 21, 2015 at 4:44 PM, Peter Lin wrote:
> I don't remember other people's examples in detail due to my shitty
> memory, so I'd rather not misquote.
>
Fair enough, but maybe you shouldn't use "people's examples you don't
remenber" as argument then. Those examples might be wrong or outda
I don't remember other people's examples in detail due to my shitty memory,
so I'd rather not misquote.
In my case, I mix static and dynamic columns in a single column family with
primitives and objects. The objects are temporal object graphs with a known
type. Doing this type of stuff is basicall
> I've chatted with several long time users of Cassandra and there's things
> CQL3 doesn't support.
>
Would you care to elaborate then? Maybe a simple example of something (or
multiple things since you used plural) in thrift that cannot be supported
in CQL?
And please note that I'm *not* saying th
I've studied the source code and I don't believe that statement is true.
I've chatted with several long time users of Cassandra and there's things
CQL3 doesn't support.
Like I've said before. Thrift and CQL3 compliment each other. I totally
understand some committers don't want the overhead due to
On Wed, Jan 21, 2015 at 3:46 AM, Peter Lin wrote:
>
> I don't understand why people [...] pretend it supports 100% of the use
> cases.
>
Have you consider the possibly that it's actually true and you're just
wrong by lack of knowledge?
--
Sylvain
Hello,
Peter highlighted the tradeoff between Thrift and CQL3 nicely in this
case, i.e. requiring a different design approach for this solution.
Collections do not sound like a good fit for your current challenge, but is
there a different way to design/solve your challenge using CQL techniques?
The original dynamic column idea in Google BigTable paper is a mapping of:
(row key, raw bytes) -> raw bytes
The restriction imposed by CQL is, as far as I understand, you need to have a
type for each column.
If the value types involved in the schema is limited, e.g. text or int or
timesta
the thing is, CQL only handles some types of dynamic column use cases.
There's plenty of examples on datastax.com that shows how to do CQL style
dynamic columns.
based on what was described by Chetan, I don't feel CQL3 is a perfect fit
for what he wants to do. To use CQL3, he'd have to change his
I think that table example misses the point of chetan's functional
requirement. he actually needs dynamic columns.
On Tue, Jan 20, 2015 at 8:12 PM, Xu Zhongxing wrote:
> Maybe this is the closest thing to "dynamic columns" in CQL 3.
>
> create table reivew (
> product_id bigint,
> create
Maybe this is the closest thing to "dynamic columns" in CQL 3.
create table reivew (
product_id bigint,
created_at timestamp,
data_key text,
data_tvalue text,
data_ivalue int,
primary key ((priduct_id, created_at), data_key)
);
data_tvalue and data_ivalue is optional.
Hi,
Adding to previous mail. For example: We have a column family named review
(with some arbitrary data in map).
CREATE TABLE review(
product_id bigint,
created_at timestamp,
data_int map,
data_text map,
PRIMARY KEY (product_id, created_at)
);
Assume that these 2 maps I use to store arbitrary d
Hi,
Most of the time I will be querying on product_id and created_at, but for
analytic I need to query almost on all column.
Multiple collections ideas is good but the only is cassandra reads a
collection entirely, what if I need a slice of it, I mean
columns for certain keys which is possible wi
Hello,
There are probably lots of options to this challenge. The more details
around your use case that you can provide, the easier it will be for this
group to offer advice.
A few follow-up questions:
- How will you query this data?
- Do your queries require filtering on specific columns ot
Hi,
I am creating a review system. for instance lets assume following are the
attibutes of system:
Review{
id bigint,
product_id bigint,
created_at timestamp,
summary text,
description text,
pros set,
cons set,
feature_rating map
etc
}
I created partition key as product_id (so that all the re
Could you please explain how we can achieve dynamic column behavior by
clustering columns.
On Wed, Jan 21, 2015 at 12:10 AM, chetan verma
wrote:
> Hi,
>
> I am creating a review system. for instance lets assume following are the
> attibutes of system:
>
> Review{
> id bigint,
> product_id bigint
Hello,
Have you looked at solving this challenge with clustering columns? Also,
please describe the problem set details for more specific advice from this
group.
Starting new projects on Thrift isn't the recommended approach.
Jonathan
[image: datastax_logo.png]
Jonathan Lacefield
Solutio
yes, thrift does have async, though I haven't had to use it yet.
Right now I'm working on adding CAS to hector followed by multi slice.
On Fri, Jun 13, 2014 at 9:01 PM, graham sanderson wrote:
> Note as I mentioned mid post, thrift also supports async nowadays (there
> was a recent discussion
Note as I mentioned mid post, thrift also supports async nowadays (there was a
recent discussion on cassandra dev and the choice was not to move to it)
I think the binary protocol is the way forward; CQL3 needs some new features,
or there need to be some other types of requests you can make over
without a doubt there's nice features of CQL3 like notifications and async.
I want to see CQL3 mature and handle all the use cases that Thrift handles
easily today. It's to everyone's benefit to work together and improve CQL3.
Other benefits of Thrift drivers today is being able to use object API
Just to add 2 more cents... :)
The CQL3 protocol is asynchronous. This can provide a substantial
throughput increase, according to my benchmarking, when one uses
non-blocking techniques.
It is also peer-to-peer. Hence the server can generate events to send to
the client, e.g. schema changes - in
My 2 cents…
A motivation for CQL3 AFAIK was to make Cassandra more familiar to SQL users.
This is a valid goal, and works well in many cases.
Equally there are use cases (that some might find ugly) where Cassandra is
chosen explicitly because of the sorts of things you can do at the thrift
leve
There are always the pros and the cons with a querying language, as always.
But as far as I can see, the advantages of Thrift I can see over CQL3 are:
1) Thrift require a little bit less decoding server-side (a difference
around 10% in CPU usage).
2) Thrift use more "compact" storage because C
With text based query approach like CQL, you loose the type with dynamic
columns. Yes, we're storing it as bytes, but it is simpler and easier with
Thrift to do these types of things.
I like CQL3 and what it does, but text based query languages make certain
dynamic schema use cases painful. Having
"the validation type is set to bytes, and my code is type safe, so it knows
which serializers to use. Those dynamic columns are driven off the types in
Java." --> Correct. However, you are still bound by the column comparator
type which should be fixed (unless again you set it to bytes, in this ca
the validation type is set to bytes, and my code is type safe, so it knows
which serializers to use. Those dynamic columns are driven off the types in
Java.
Having said that, CQL3 does have a new custom type feature, but the
documentation is basically non-existent on how that actually works. One
c
In thrift, when creating a column family, you need to define
1) the row/partition key type
2) the column comparator type
3) the validation type for the actual value (cell in CQL3 terminology)
Unless you use "dynamic composites" feature, which does not exist (and
probably won't) in CQL3, I don't s
when I say dynamic column, I mean non-static columns of different types
within the same row. Some could be an object or one of the defined
datatypes.
with thrift I use the appropriate serializer to handle these dynamic
columns.
On Fri, Jun 13, 2014 at 4:55 PM, DuyHai Doan wrote:
> Well, before
Well, before talking and discussing about "dynamic columns", we should
first define it clearly. What do people mean by "dynamic columns" exactly ?
Is it the ability to add many columns "of same type" to an existing
physical row? If yes then CQL3 does support it with clustering columns.
On Fri, J
Yeah I don't anticipate more than 1000 properties, well under in fact. I
guess the trade off of using the clustered columns is that I'd have a table
that would be tall and skinny which also has its challenges w/r/t memory.
I'll look into your suggestion a bit more and consider some others around a
Hi Mark
I believe that in your table you want to have some "common" fields that
will be there whatever customer is, and other fields that are entirely
customer-dependent, isn't it ?
In this case, creating a table with static columns for the common fields
and a clustering column representing all
Like you, I make extensive use of dynamic columns for similar reasons.
In our project, one of the goals is to give "end users" the ability to
design their own schema without having to alter a table. If people really
want strong schema, then just use old Sql or NewSql. RDB gives you the full
power
My use case requires the support of arbitrary columns much like a CRM. My
users can define 'custom' fields within the application. Ideally I wouldn't
have to change the schema at all, which is why I like the old thrift
approach rather than the CQL approach.
Having said all that, I'd be willing to
I like CQL, but it's not a hammer.
If thrift is more appropriate for you, then use it. If Cassandra gets to
the point where Thrift is removed, I'll just fork Cassandra. That's what's
great about open source.
On Fri, Jun 13, 2014 at 3:47 PM, DuyHai Doan wrote:
> This strikes me as bad practice
This strikes me as bad practice in the world of multi tenant systems. I
don't want to create a table per customer. So I'm wondering if dynamically
modifying the table is an accepted practice? --> Can you give some details
about your use case ? How would you "alter" a table structure to adapt it
to
On Fri, Jun 13, 2014 at 11:54 AM, DuyHai Doan wrote:
> Dynamic columns, as you said, are perfectly supported by CQL3 via
> clustering columns.
>
"Perfectly supported" seems a bit expansive as a claim. They are not quite
the same thing as actual dynamic columns and are meaningfully worse in some
Thanks DuyHai,
I have a follow up question to #2. You mentioned ideally I would create a
new table instead of mutating an existing one.
This strikes me as bad practice in the world of multi tenant systems. I
don't want to create a table per customer. So I'm wondering if dynamically
modifying the
Hello Mark
Dynamic columns, as you said, are perfectly supported by CQL3 via
clustering columns. And no, using collections for storing dynamic data is a
very bad idea if the cardinality is very high (>> 1000 elements)
1) Is using Thrift a valid approach in the era of CQL? --> Less and less.
Un
Hey, this is good info. Seems like i have the same capabilities, i just
need to twist my brain a bit to see it better. Thanks for all the
feedback, much appreciated.
Joe
On Thu, Jun 6, 2013 at 11:27 AM, Alain RODRIGUEZ wrote:
> Not sure if you remember this Jonathan, but Sylvain already wrot
Not sure if you remember this Jonathan, but Sylvain already wrote a very
clear documentation about it :
http://www.datastax.com/dev/blog/thrift-to-cql3 (OCTOBER 26, 2012)
Yet a second page will give to this important topic a greater visibility.
2013/6/6 Jonathan Ellis
> This is becoming someth
Your data model should take into consideration the number of items you're
storing in a collection. If you expect it will grow over time with no
small upper bound, don't use a collection. You don't need to read before
write to answer this question, it's a decision made at modeling time
(before you
This is becoming something of a FAQ, so I wrote an more in-depth
answer:
http://www.datastax.com/dev/blog/does-cql-support-dynamic-columns-wide-rows
On Thu, Jun 6, 2013 at 8:02 AM, Joe Greenawalt wrote:
> Hi,
> I'm having some problems figuring out how to append a dynamic column on a
> column fa
The problem about "being careful about how much you store in a collection"
is that Cassandra is a blind-write system. Knowing how much data is
currently in the collection before you write is an anti-pattern, read
before write.
Cassandra Rule 1: DON'T READ BEFORE WRITE
Cassandra Rule 2: ROWS CAN HA
>
> CQL3 does now support dynamic columns. For tags or metadata values you
> could use a Collection:
>
This should probably be clarified. A collection is a super useful tool,
but it is *not* the same thing as a dynamic column. It has many
advantages, but there is one huge disadvantage in that yo
Hi,
CQL3 does now support dynamic columns. For tags or metadata values you could
use a Collection:
http://www.datastax.com/dev/blog/cql3_collections
For wide rows there's the enhanced primary keys, which I personally prefer over
the composite columns of yore:
http://www.datastax.com/dev/blog/
Dynamic columns are not supported in CQL3. We just had a discussion a day
or two ago about this where Eric Stevens explained it. Please see this:
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/CQL-3-returning-duplicate-keys-td7588181.html
Regards,
Shahab
On Thu, Jun 6, 2013 at
Yes.
(Please keep these to the user list rather than dev.)
On Fri, Jan 6, 2012 at 11:59 AM, Frank Yang wrote:
> Hi everyone,
>
> I am wondering whether it is possible to not to define the column
> metadata when creating a column family, but to specify the column when
> client updates data, for e
57 matches
Mail list logo
