Re: cluster size, several cluster on one node for multi-tenancy

2011-02-17 Thread Mimi Aluminium
Hi,
I really need your help in this matter.
I will try to simplify my problem and ask specific questions

I am thinking of solving the multi-tenancy problem by providing a separate
cluster per each tenant. Does it sound reasonable?
I can end-up with one node belongs to several clusters.
Does Cassandra support several clusters per node? Does it mean several
Cassandra daemons on each node? Do you recommend doing that ? what is the
overhead? is there any link that explain how to do that?

Thanks a lot,
Mimi


On Wed, Feb 16, 2011 at 6:43 PM, Mimi Aluminium wrote:

>  Hi,
> We are interested in a multi-tenancy environment, that may consist of up to
> hundreds of data centers. The current design requires cross rack and cross
> DC replication. Specifically, the per-tenant CFs will be replicated 6 times:
> in three racks,  with 2 copies inside a rack, the racks will be located in
> at least two different DCs. In the future other replication policies will be
> considered. The application will decide where (which racks and DC)  to place
> each tenant's replicas.  and it might be that one rack can hold more than
> one tenant.
>
> Separating each tenant in a different keyspace, as was suggested
> in  previous mail thread in this subject, seems to be a good approach
> (assuming the memtable problem will be solved somehow).
> But then we had concern with regard to the cluster size.
> and here are my questions:
> 1) Given the above, should I define one Cassandra cluster that hold all the
> DCs? sounds not reasonable  given hundreds DCs tens of servers in each DC
> etc. Where is the bottleneck here? keep-alive messages, the gossip, request
> routing? what is the largest number of servers a cluster can bear?
> 2) Now assuming that I can create the per-tenant  keyspace only for  the
> servers that in the three racks where the replicas are held,  does such
> definition reduces the messaging transfer among the other servers. Does
> Cassandra optimizes the message transfer in such case?
> 3) Additional possible solution was to create a separate clusters per each
> tenant. But it can cause a situation where one server has to run two or more
> Cassandra's clusters. Can we run more than one cluster in parallel, does it
> means two cassandra daemons / instances on one server? what will be the
> overhead? do you have a link that explains how to deal with it?
>
> Please can you help me to decide which of these solution can work or you
> are welcome to suggest something else.
> Thanks a lot,
> Mimi
>
>
>
>
>
>
>
>


node failure, and automatic decommission (or removetoken)

2011-02-27 Thread Mimi Aluminium
Hi,
I have a question about a tool or a wrapper that perform automatic data move
upon node failure?
Assuming I have  3 nodes with a replication factor of 3. In case of one node
failure, does the third replica (that was located before on the failed node
) re-appears on one the of live nodes?
I am looking for something that is similar to Hinted Handoff but with with a
viable that can be read.
I know we can stream manually the data (using nodetool move or
decommissions), but is there something automatic?
I also found an open ticket 957 but was not sure this is what I am looking
for.
Thanks
Miriam




Miriam Allalouf, PhD

Storage & Network Research
IBM, Haifa Research Labs
Tel: 972-3-7689525
Mobile: 972-52-3664129
e-mail: miri...@il.ibm.com


disk (sstable file, index and BF) layout

2011-03-01 Thread Mimi Aluminium
Hi,
Can you please help me in modeling the disk layout.
I am estimating  the size of the sstable file, index and BF (for a certain
data model) using the details in ths link
http://wiki.apache.org/cassandra/ArchitectureSSTable and compare it with the
real DB files.
Unfortunately, there is a mismatch between the estimated size and real file.
Does the timestamp field is allocated even if no value was assigned to it?
The real index file row is much larger than what is described, and the
structure of the column index is not clear.
Can you give me more details about the index file, column index and BF size?

Please can you refer me to the code where those field are written to the
disk?
Thanks a lot,
Miriam