After check the log and configuration, I found it caused by two reason.
1. GC grace seconds
I using hector client to connect cassandra, and the default value of GC
grace seconds for each column family is **Zero** ! So when hinted handoff
replay the temporary value, the tombstone on other two
I have a 3 nodes cluster, replicate_factor is 3 also. Consistency level is
Write quorum, Read quorum.
Traffic has three major steps
Create:
Rowkey:
Column: status=new, requests="x"
Update:
Rowkey:
Column: status=executing, requests="x"
Delete:
Rowkey: