Re: Recreating materialized views in cassandra

2020-07-29 Thread Jasonstack Zhao Yang
> Also, do you think putting the '-Dmv_enable_coordinator_batchlog=true' parameter in cassandra.yaml will solve or reduce the issue to some extent? It should improve the eventual consistency of MV. Only enable it if there is enough CPU/IO capacity in the cluster. Do you mind creating a JIRA descr

Re: Recreating materialized views in cassandra

2020-07-29 Thread Jasonstack Zhao Yang
> The cluster started to crash when some partitions in MV crossed 1 GB size at few nodes, whereas in other nodes it is less than 50 MB. > Should we be worried about this? Depend on your MV partition key design. The memory pressure of wide partitions is improved in 3.x (CASSANDRA-11206

Re: Recreating materialized views in cassandra

2020-07-27 Thread Saijal Chauhan
>do you run "nodetool repair" on both base and view regularly? Yes, we run a full repair on our entire cluster every weekend which includes the keyspaces with the base table and materialized views But still, there are a ton of discrepancies in our base table and materialized view. Also, do you th

Re: Recreating materialized views in cassandra

2020-07-27 Thread Jasonstack Zhao Yang
Hi, > We are facing data inconsistency issues between base tables and materialized views. do you run "nodetool repair" on both base and view regularly? > What are all the possible scenarios that we should be watching out for in a production environment? more cpu/io/gc for populating views. > C

Recreating materialized views in cassandra

2020-07-27 Thread Saijal Chauhan
Hi, We are using Cassandra 3.0.13 We have the following datacenters: - DC1 with 7 Cassandra nodes with RF:3 - DC2 with 2 Cassandra nodes with RF:2 - DC3 with 2 Cassandra nodes with RF:2 We are facing data inconsistency issues between base tables and materialized views. The only solution