Re: high write latency on a single table

2019-07-26 Thread CPC
In the mean time we opened traces and found that normal insert operations does not have high latency. Delete operations have high latency. And all deletes are range deletes. Is there any performance regression about range deletes? Mean time of delete operations are 45 times higher than insert opera

Re: high write latency on a single table

2019-07-25 Thread mehmet bursali
awesome! so we can make a further investigation by using cassandra exporter on this link.  https://github.com/criteo/cassandra_exporter This exporter gives detailed information for read/write operations on each column  by using metrics below.. org:apache:cassandra:metrics:columnfamily:.* ( read

Re: high write latency on a single table

2019-07-24 Thread CPC
Hi Mehmet, Yes prometheus and opscenter On Wed, 24 Jul 2019 at 17:09, mehmet bursali wrote: > hi, > do you use any perfomance monitoring tool like prometheus? > > > > > On Monday, July 22, 2019, 1:16:58 PM GMT+3, CPC > wrote: > > > Hi everybody, > > State column contains "R" or "D" values. Jus

Re: high write latency on a single table

2019-07-24 Thread mehmet bursali
hi,do you use any perfomance monitoring tool like prometheus? On Monday, July 22, 2019, 1:16:58 PM GMT+3, CPC wrote: Hi everybody, State column contains "R" or "D" values. Just a single character. As Rajsekhar said, only difference is the table can contain high number of cell count.

Re: high write latency on a single table

2019-07-22 Thread CPC
Hi everybody, State column contains "R" or "D" values. Just a single character. As Rajsekhar said, only difference is the table can contain high number of cell count. In the mean time we made a major compaction and data per node was 5-6 gb. On Mon, Jul 22, 2019, 10:56 AM Rajsekhar Mallick wrote:

Re: high write latency on a single table

2019-07-22 Thread Rajsekhar Mallick
Hello Team, The difference in write latencies between both the tables though significant,but the higher latency being 11.353 ms is still acceptable. Overall Writes not being an issue, but write latency for this particular table on the higher side does point towards data being written to the table

Re: high write latency on a single table

2019-07-22 Thread Ben Slater
Is the size of the data in your “state” column variable? The higher write latencies at the 95%+ could line up with large volumes of data for particular rows in that column (the one column not in both tables)? Cheers Ben --- *Ben Slater**Chief Product Officer*

Re: high write latency on a single table

2019-07-21 Thread CPC
Hi guys, Any idea? I thought it might be a bug but could not find anything related on jira. On Fri, Jul 19, 2019, 12:45 PM CPC wrote: > Hi Rajsekhar, > > Here the details: > > 1) > > [cassadm@bipcas00 ~]$ nodetool tablestats tims.MESSAGE_HISTORY > Total number of tables: 259 >

Re: high write latency on a single table

2019-07-19 Thread CPC
Hi Rajsekhar, Here the details: 1) [cassadm@bipcas00 ~]$ nodetool tablestats tims.MESSAGE_HISTORY Total number of tables: 259 Keyspace : tims Read Count: 208256144 Read Latency: 7.655146714749506 ms Write Count: 2218205275 Write Latency: 1.7826005

Re: high write latency on a single table

2019-07-18 Thread Rajsekhar Mallick
Hello, Kindly post below details 1. Nodetool cfstats for both the tables. 2. Nodetool cfhistograms for both the tables. 3. Replication factor of the tables. 4. Consistency with which write requests are sent 5. Also the type of write queries for the table if handy would also help (Light weight t

high write latency on a single table

2019-07-18 Thread CPC
Hi all, Our cassandra cluster consist of two dc and every dc we have 10 nodes. We are using DSE 5.1.12 (cassandra 3.11).We have a high local write latency on a single table. All other tables in our keyspace have normal latencies like 0.02 msec,even tables that have more write tps and more data. B