Adding new Datacenter

2018-12-05 Thread Rajasekhar Kommineni
Hello everyone, I am adding new DC to my existing cluster, with application using consistency of ONE. Will the new node of new DC participates in serving the requests during the bootstrapping/ rebuild ? I tested the scenario of building lost seed node with nodetool rebuild , where the binary se

Re: Adding new datacenter to the cluster

2018-08-20 Thread Oleksandr Shulgin
On Mon, Aug 13, 2018 at 3:50 PM Vitali Dyachuk wrote: > Hello, > I'm going to follow this documentation to add a new datacenter to the C* > cluster > > https://docs.datastax.com/en/cassandra/3.0/cassandra/operations/opsAddDCToCluster.html > > The main step is to run nodetool rebuild which will sy

Adding new datacenter to the cluster

2018-08-13 Thread Vitali Dyachuk
Hello, I'm going to follow this documentation to add a new datacenter to the C* cluster https://docs.datastax.com/en/cassandra/3.0/cassandra/operations/opsAddDCToCluster.html The main step is to run nodetool rebuild which will sync data to the new datacenter, this will load cluster badly since the

Re: Rebuild failing while adding new datacenter

2016-10-22 Thread Jai Bheemsen Rao Dhanwada
I have opened https://issues.apache.org/jira/browse/CASSANDRA-12816 On Thu, Oct 20, 2016 at 6:30 PM, Yabin Meng wrote: > Sorry, I'm not aware of it > > On Thu, Oct 20, 2016 at 6:00 PM, Jai Bheemsen Rao Dhanwada < > jaibheem...@gmail.com> wrote: > >> Thank you Yabin, is there a exisiting JIRA th

Re: Rebuild failing while adding new datacenter

2016-10-20 Thread Yabin Meng
Sorry, I'm not aware of it On Thu, Oct 20, 2016 at 6:00 PM, Jai Bheemsen Rao Dhanwada < jaibheem...@gmail.com> wrote: > Thank you Yabin, is there a exisiting JIRA that I can refer to? > > On Thu, Oct 20, 2016 at 2:05 PM, Yabin Meng wrote: > >> I have seen this on other releases, on 2.2.x. The wo

Re: Rebuild failing while adding new datacenter

2016-10-20 Thread Jai Bheemsen Rao Dhanwada
Thank you Yabin, is there a exisiting JIRA that I can refer to? On Thu, Oct 20, 2016 at 2:05 PM, Yabin Meng wrote: > I have seen this on other releases, on 2.2.x. The workaround is exactly > like yours, some other system keyspaces also need similar changes. > > I would say this is a benign bug.

Re: Rebuild failing while adding new datacenter

2016-10-20 Thread Yabin Meng
I have seen this on other releases, on 2.2.x. The workaround is exactly like yours, some other system keyspaces also need similar changes. I would say this is a benign bug. Yabin On Thu, Oct 20, 2016 at 4:41 PM, Jai Bheemsen Rao Dhanwada < jaibheem...@gmail.com> wrote: > thanks, > > This alway

Re: Rebuild failing while adding new datacenter

2016-10-20 Thread Jai Bheemsen Rao Dhanwada
thanks, This always works on 2.1.13 and 2.1.16 version but not on 3.0.8. definitely not a firewall issue On Thu, Oct 20, 2016 at 1:16 PM, sai krishnam raju potturi < pskraj...@gmail.com> wrote: > we faced a similar issue earlier, but that was more related to firewall > rules. The newly added dat

Re: Rebuild failing while adding new datacenter

2016-10-20 Thread sai krishnam raju potturi
we faced a similar issue earlier, but that was more related to firewall rules. The newly added datacenter was not able to communicate with the existing datacenters on the port 7000(inter-node communication). Your's might be a different issue, but just saying. On Thu, Oct 20, 2016 at 4:12 PM, Jai

Rebuild failing while adding new datacenter

2016-10-20 Thread Jai Bheemsen Rao Dhanwada
Hello All, I have single datacenter with 3 C* nodes and we are trying to expand the cluster to another region/DC. I am seeing the below error while doing a "nodetool rebuild -- name_of_existing_data_center" . [user@machine ~]$ nodetool rebuild DC1 nodetool: Unable to find sufficient sources for s

Re: Rebuild failing when adding new datacenter (3.0.8)

2016-09-22 Thread Timo Ahokas
Hi Yabin/Alain, I changed the replication strategies for system_distributed, system_auth and system_traces to use NetworkTopologyStrategies and repaired the affected keyspaces. Now the rebuild process starts up ok without errors. Thanks a lot for your help! Best regards, Timo On 22 September 20

Re: Rebuild failing when adding new datacenter (3.0.8)

2016-09-22 Thread Yabin Meng
It is a Cassandra bug. The workaround is to change system_distributed keyspce replication strategy to something as below: alter keyspace system_distributed with replication = {'class': 'NetworkTopologyStrategy', 'DC1': '3', 'DC2': '3', 'DC3': '3'}; You may see similar problem for other sys

Re: Rebuild failing when adding new datacenter (3.0.8)

2016-09-22 Thread Timo Ahokas
Hi Alain, Our normal user keyspaces have RF3 in all DCs, e.g: create keyspace reporting with replication = {'class': 'NetworkTopologyStrategy', 'DC1': '3', 'DC2': '3', 'DC3': '3'}; Any idea would it be safe to change the system_distributed keyspace to match this? -Timo On 22 September 2016 at

Re: Rebuild failing when adding new datacenter (3.0.8)

2016-09-22 Thread Timo Ahokas
Hi Alain, Thanks a lot for a helping out! Some of the basic keyspace / cluster info you requested: # echo "DESCRIBE KEYSPACE system_distributed;" | cqlsh CREATE KEYSPACE system_distributed WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '3'} AND durable_writes = true; CRE

Re: Rebuild failing when adding new datacenter (3.0.8)

2016-09-22 Thread Alain RODRIGUEZ
It could be a bug. Yet I am not very aware of this system_distributed keyspace, but from what I see, it is using a simple strategy: root@tlp-cassandra-2:~# echo "DESCRIBE KEYSPACE system_distributed;" | cqlsh $(hostname -I | awk '{print $1}') CREATE KEYSPACE system_distributed WITH replication =

Rebuild failing when adding new datacenter (3.0.8)

2016-09-22 Thread Timo Ahokas
Hi, We have a Cassandra 3.0.8 cluster (recently upgraded from 2.1.15) currently running in two data centers (13 and 19 nodes, RF3 in both). We are adding a third data center before decommissioning one of the earlier ones. Installing Cassandra (3.0.8) goes fine and all the nodes join the cluster (n