Re: repair failed

2020-01-02 Thread Ben Mills
6 1565 > > > - 原始邮件 - > 发件人:Oliver Herrmann > 收件人:user@cassandra.apache.org > 主题:repair failed > 日期:2019年12月28日 23点15分 > > Hello, > > today the second time our weekly repair job failed which was working for > many month without a problem. We are having mu

回复:repair failed

2019-12-29 Thread gloCalHelp.com
TO Oliver : Maybe repair should be executed after all data in MEMTBL are all flushed into harddisk? Sincerely yours, Georgelin www_8ems_...@sina.com mobile:0086 180 5986 1565 - 原始邮件 - 发件人:Oliver Herrmann 收件人:user@cassandra.apache.org 主题:repair failed 日期:2019年12月28日 23点15分

repair failed

2019-12-28 Thread Oliver Herrmann
if the repair is started only on one node? The repair fails after one hour with the following error message: failed with error Could not create snapshot at /192.168.13.232 (progress: 0%) [2019-12-28 05:00:04,295] Some repair failed [2019-12-28 05:00:04,296] Repair command #1 finished in 1 hour 0

Re: Repair failed and crash the node, how to bring it back?

2019-08-01 Thread Martin Xue
ested with lots of > tombstones which in turn also tax on heap consumption. My $.002 cents for > the moment. > > > > > > > > *From:* Martin Xue [mailto:martin...@gmail.com] > *Sent:* Wednesday, July 31, 2019 5:05 PM > *To:* user@cassandra.apache.org > *Subject:*

Re: Repair failed and crash the node, how to bring it back?

2019-08-01 Thread Martin Xue
Hi Alex, Thanks, much appreciated. Regards Martin On Thu, Aug 1, 2019 at 3:34 PM Alexander Dejanovski wrote: > Hi Martin, > > apparently this is the bug you've been hit by on hints : > https://issues.apache.org/jira/browse/CASSANDRA-14080 > It was fixed in 3.0.17. > > You didn't provide the l

RE: Repair failed and crash the node, how to bring it back?

2019-08-01 Thread ZAIDI, ASAD A
:05 PM To: user@cassandra.apache.org Subject: Re: Repair failed and crash the node, how to bring it back? Hi Alex, Thanks for your reply. The disk space was around 80%. The crash happened during repair, primary range full repair on 1TB keyspace. Would that crash again? Thanks Regards Martin On T

Re: Repair failed and crash the node, how to bring it back?

2019-07-31 Thread Alexander Dejanovski
Hi Martin, apparently this is the bug you've been hit by on hints : https://issues.apache.org/jira/browse/CASSANDRA-14080 It was fixed in 3.0.17. You didn't provide the logs from Cassandra at the time of the crash, only the output of nodetool, so it's hard to say what caused it. You may be hit by

Re: Repair failed and crash the node, how to bring it back?

2019-07-31 Thread Martin Xue
Hi Alex, Thanks for your reply. The disk space was around 80%. The crash happened during repair, primary range full repair on 1TB keyspace. Would that crash again? Thanks Regards Martin On Thu., 1 Aug. 2019, 12:04 am Alexander Dejanovski, wrote: > It looks like you have a corrupted hint file.

Re: Repair failed and crash the node, how to bring it back?

2019-07-31 Thread Alexander Dejanovski
It looks like you have a corrupted hint file. Did the node run out of disk space while repair was running? You might want to move the hint files off their current directory and try to restart the node again. Since you'll have lost mutations then, you'll need... to run repair ¯\_(ツ)_/¯ ---

Repair failed and crash the node, how to bring it back?

2019-07-31 Thread Martin Xue
Hi, I am running repair on production, started with one of 6 nodes in the cluster (3 nodes in each of two DC). Cassandra version 3.0.14. running: repair -pr --full keyspace on node 1, 1TB data, takes two days, and crash, error shows: 3202]] finished (progress: 3%) Exception occurred during clean

Re: repair failed or stopped after 7-8 hours?

2010-07-28 Thread Aaron Morton
Did you start the repair on all nodes at once or one at a time ? Take a look at the streams on the nodes, using either nodetool -h localhost -p 8080 streams Or the JMX interface. Check if the numbers are changing. AaronOn 28 Jul, 2010,at 08:14 AM, Michael Andreasen wrote:I've started repair on 6 n

repair failed or stopped after 7-8 hours?

2010-07-27 Thread Michael Andreasen
I've started repair on 6 nodes some 7-8 hours ago The nodes still have load of 2-3 (normally 0.5) and if i grep AE in system.log i get lines like this on most of the nodes Performing streaming repair of 30 ranges to /172.19.0.32 for Load is 400-500gb on the nodes. Any word of advise