RE: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Kenneth Brotman
@cassandra.apache.org Subject: RE: 答复: 答复: A node down every day in a 6 nodes cluster If you think that will fix the problem, maybe you could add a little more memory to each machine as a short term fix. From: Xiangfei Ni [mailto:xiangfei...@cm-dt.com] Sent: Wednesday, March 28, 2018 5:24 AM To: user

RE: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Kenneth Brotman
2516 发件人: Kenneth Brotman 发送时间: 2018年3月28日 20:16 收件人: user@cassandra.apache.org 主题: RE: 答复: 答复: A node down every day in a 6 nodes cluster David, Did you figure out what to do about the data model problem? It could be that your data files finally grow to the point that the data

答复: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Xiangfei Ni
Brotman 发送时间: 2018年3月28日 20:16 收件人: user@cassandra.apache.org 主题: RE: 答复: 答复: A node down every day in a 6 nodes cluster David, Did you figure out what to do about the data model problem? It could be that your data files finally grow to the point that the data model problem caused the Java heap

RE: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Kenneth Brotman
model. Kenneth Brotman From: Kenneth Brotman [mailto:kenbrot...@yahoo.com] Sent: Wednesday, March 28, 2018 4:46 AM To: 'user@cassandra.apache.org' Subject: RE: 答复: 答复: A node down every day in a 6 nodes cluster Was any change to hardware done around the time the problem star

RE: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Kenneth Brotman
-dt.com] Sent: Wednesday, March 28, 2018 4:40 AM To: user@cassandra.apache.org Subject: 答复: 答复: 答复: A node down every day in a 6 nodes cluster Hi Kenneth, The cluster has been running for 4 months, The problem occurred from last week, Best Regards, 倪项菲/ David Ni 中移德电网络科技有限公司

答复: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Xiangfei Ni
: Kenneth Brotman 发送时间: 2018年3月28日 19:34 收件人: user@cassandra.apache.org 主题: RE: 答复: 答复: A node down every day in a 6 nodes cluster David, How long has the cluster been operating? How long has the problem been occurring? Kenneth Brotman From: Jeff Jirsa [mailto:jji...@gmail.com] Sent: Tuesday, March 27

RE: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-28 Thread Kenneth Brotman
Jeff Jirsa 发送时间: 2018年3月27日 11:03 收件人: user@cassandra.apache.org 主题: Re: A node down every day in a 6 nodes cluster That warning isn’t sufficient to understand why the node is going down Cassandra 3.9 has some pretty serious known issues - upgrading to 3.11.3 is likely a good idea

Re: 答复: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Jeff Jirsa
Virtue Intelligent Network Ltd, co. > > Add: 2003,20F No.35 Luojia creative city,Luoyu Road,Wuhan,HuBei > Mob: +86 13797007811|Tel: + 86 27 5024 2516 > > 发件人: Jeff Jirsa > 发送时间: 2018年3月27日 11:03 > 收件人: user@cassandra.apache.org > 主题: Re: A node down every day in

答复: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Xiangfei Ni
,Wuhan,HuBei Mob: +86 13797007811|Tel: + 86 27 5024 2516 发件人: Xiangfei Ni 发送时间: 2018年3月28日 9:45 收件人: Jeff Jirsa 抄送: user@cassandra.apache.org 主题: 答复: 答复: A node down every day in a 6 nodes cluster Hi Jeff, Today another node was shutdown,I have attached the exception log file,could you

答复: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Xiangfei Ni
: + 86 27 5024 2516 发件人: Jeff Jirsa 发送时间: 2018年3月27日 11:50 收件人: Xiangfei Ni 抄送: user@cassandra.apache.org 主题: Re: 答复: A node down every day in a 6 nodes cluster Only one node having the problem is suspicious. May be that your application is improperly pooling connections, or you have a hardware

RE: RE: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Rahul Singh
m/en/cassandra/3.0/cassandra/operations/opsReplaceLiveNode.html > > Kenneth Brotman > > > From: Xiangfei Ni [mailto:xiangfei...@cm-dt.com] > Sent: Tuesday, March 27, 2018 5:44 AM > To: user@cassandra.apache.org > Subject: Re:RE: 答复: A node down every day in a 6 nodes clu

RE: RE: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Kenneth Brotman
/operations/opsReplaceLiveNode.html Kenneth Brotman From: Xiangfei Ni [mailto:xiangfei...@cm-dt.com] Sent: Tuesday, March 27, 2018 5:44 AM To: user@cassandra.apache.org Subject: Re:RE: 答复: A node down every day in a 6 nodes cluster Thanks,Kenneth,this is production database,and it is

Re:RE: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Xiangfei Ni
[mailto:xiangfei...@cm-dt.com] Sent: Tuesday, March 27, 2018 3:27 AM To: Jeff Jirsa Cc: user@cassandra.apache.org Subject: 答复: 答复: A node down every day in a 6 nodes cluster Thanks Jeff, So your suggestion is to first resolve the data model issue which cause wide partition,right? Best Regards

RE: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Kenneth Brotman
13797007811|Tel: + 86 27 5024 2516 发件人: Jeff Jirsa 发送时间: 2018年3月27日 11:50 收件人: Xiangfei Ni 抄送: user@cassandra.apache.org 主题: Re: 答复: A node down every day in a 6 nodes cluster Only one node having the problem is suspicious. May be that your application is improperly pooling connections, or you

答复: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Xiangfei Ni
5024 2516 发件人: Jeff Jirsa 发送时间: 2018年3月27日 11:50 收件人: Xiangfei Ni 抄送: user@cassandra.apache.org 主题: Re: 答复: A node down every day in a 6 nodes cluster Only one node having the problem is suspicious. May be that your application is improperly pooling connections, or you have a hardware problem. I

答复: 答复: A node down every day in a 6 nodes cluster

2018-03-27 Thread Xiangfei Ni
@cassandra.apache.org 主题: Re: 答复: A node down every day in a 6 nodes cluster Only one node having the problem is suspicious. May be that your application is improperly pooling connections, or you have a hardware problem. I dont see anything in nodetool that explains it, though you certainly have a data

答复: 答复: A node down every day in a 6 nodes cluster

2018-03-26 Thread Xiangfei Ni
27 5024 2516 发件人: daemeon reiydelle 发送时间: 2018年3月27日 11:42 收件人: user 主题: Re: 答复: A node down every day in a 6 nodes cluster Look for errors on your network interface. I think you have periodic errors in your network connectivity <==> "Who do you think made the first stone

Re: 答复: A node down every day in a 6 nodes cluster

2018-03-26 Thread Jeff Jirsa
+86 13797007811 <+86%20137%209700%207811>|Tel: + 86 27 5024 2516 > <+86%2027%205024%202516> > > > > *发件人:* Jeff Jirsa > *发送时间:* 2018年3月27日 11:03 > *收件人:* user@cassandra.apache.org > *主题:* Re: A node down every day in a 6 nodes cluster > > > > That wa

Re: 答复: A node down every day in a 6 nodes cluster

2018-03-26 Thread daemeon reiydelle
lt;+86%2027%205024%202516> > > > > *发件人:* Jeff Jirsa > *发送时间:* 2018年3月27日 11:03 > *收件人:* user@cassandra.apache.org > *主题:* Re: A node down every day in a 6 nodes cluster > > > > That warning isn’t sufficient to understand why the node is going down > > >

Re: A node down every day in a 6 nodes cluster

2018-03-26 Thread Jeff Jirsa
That warning isn’t sufficient to understand why the node is going down Cassandra 3.9 has some pretty serious known issues - upgrading to 3.11.3 is likely a good idea Are the nodes coming up on their own? Or are you restarting them? Paste the output of nodetool tpstats and nodetool cfstats

A node down every day in a 6 nodes cluster

2018-03-26 Thread Xiangfei Ni
Hi Cassandra experts, I am facing an issue,a node downs every day in a 6 nodes cluster,the cluster is just in one DC, Every node has 4C 16G,and the heap configuration is MAX_HEAP_SIZE=8192m HEAP_NEWSIZE=512m,every node load about 200G data,the RF for the business CF is 3,a node downs one tim