Re: Switching to Incremental Repair

2024-02-15 Thread Chris Lohfink
I would recommend adding something to C* to be able to flip the repaired state on all sstables quickly (with default OSS can turn nodes off one at a time and use sstablerepairedset). It's a life saver to be able to revert back to non-IR if migration going south. Same can be used to quickly switch i

Re: Nodetool command to pre-load the chunk cache

2023-03-24 Thread Chris Lohfink
Something additional to consider (outside C* fix) is using a tool like happycache <https://github.com/hashbrowncipher/happycache> to have consistent pagecache between them. Might be sufficient if the data is in memory already. Chris On Tue, Mar 21, 2023 at 2:48 PM Jeff Jirsa wrote:

[Request] End user comments for Apache Cassandra 4.1

2022-07-27 Thread Chris Thornett
Hey everyone, We're pulling together comments from end users on the release of Apache Cassandra 4.1 (coming soon). If you would like to contribute, please email them to me on *chris at constantia dot io*. Essentially, we're looking for positive quotes on your decision to use Cas

[Marketing] Share your ApacheCon experiences

2022-05-12 Thread Chris Thornett
ns. Thanks! -- Chris Thornett senior content strategist, Constantia.io ch...@constantia.io

Interview opportunity for Cassandra users

2022-01-10 Thread Chris Thornett
Hello everyone, My name is Chris, and I provide content support for the Apache Cassandra project. As part of Apache Cassandra's ongoing content marketing, I'd like to give Cassandra users the opportunity to participate in a little interview series called 'Inside Cassandra&

Re: Performance drop of current Java drivers

2020-05-01 Thread Chris Splinter
Hi Matthias, I have forwarded this to the developers that work on the Java driver and they will be looking into this first thing next week. Will circle back here with findings, Chris On Fri, May 1, 2020 at 12:28 AM Erick Ramirez wrote: > Matthias, I don't have an answer to your ques

Re: COPY command with where condition

2020-01-17 Thread Chris Splinter
t; I don't really know for the moment in production environment, but for > developpment environment the table contains more than 10.000.000 rows. > But we need just a sub dataset of this table not the entirety ... > -- > *De :* Chris Splinter

Re: COPY command with where condition

2020-01-17 Thread Chris Splinter
t; > } > } > advanced { > > auth-provider { > class = PlainTextAuthProvider > username = "superuser" > password = "mypass" > > } > } > } > -- > *

Re: COPY command with where condition

2020-01-17 Thread Chris Splinter
DSBulk has an option that lets you specify the query ( including a WHERE clause ) See Example 19 in this blog post for details: https://www.datastax.com/blog/2019/06/datastax-bulk-loader-unloading On Fri, Jan 17, 2020 at 7:34 AM Jean Tremblay < jean.tremb...@zen-innovations.com> wrote: > Did you

Unified DataStax drivers

2020-01-16 Thread Chris Splinter
x.com/blog/2019/12/tools-for-apache-cassandra> that we made available for use with Apache Cassandra in December last year, we hope that this helps simplify the picture for those that use our drivers. Best, Chris

Re: Replication system_distributed

2020-01-10 Thread Chris Splinter
ted is not a good idea for the same reasons why changing the replication of *any* keyspace to 1 is not a good idea. You lose the ability to query that data if a single node goes down. Hope this helps, Chris On Wed, Jan 8, 2020 at 1:23 AM Marcel Jakobi wrote: > Hi, > > > > the de

Re: oversized partition detection ? monitoring the partitions growth ?

2019-11-01 Thread Chris Lohfink
stable {} Chris On Thu, Oct 31, 2019 at 8:01 AM Eric LELEU wrote: > Hi, > > I'm not sure that your are able to log which partition has reached 100MB > but you may monitor the "EstimatedPartitionSizeHistogram" and take the > max value (or 99ct, 95ct) to trigger an alert us

Re: GC Tuning https://thelastpickle.com/blog/2018/04/11/gc-tuning.html

2019-10-19 Thread Chris Lohfink
others in all scenarios unfortunately. If your JVM supports it, ZGC or Shenandoah are likely going to give you the best latencies. Chris On Fri, Oct 18, 2019 at 8:41 PM Sergio Bilello wrote: > Hello! > > Is it still better to use ParNew + CMS Is it still better than G1GC th

Re: loosing data during saving data from java

2019-10-19 Thread Chris Lohfink
ill be notified of via a WriteFailure, WriteTimeout, or an OperationTimeout). A simple write like that can be idempotent so you can just try again on failure. Chris On Sat, Oct 19, 2019 at 1:26 AM adrien ruffie wrote: > Thank Jeff 🙂 > > but if you save several data to fast wi

Re: Collecting Latency Metrics

2019-05-30 Thread Chris Lohfink
sters so the loss in accuracy kinda moot. Your average for local reads/writes will almost always be sub millisecond but you might end up having 500 millisecond requests or worse that the mean will hide. Chris On Thu, May 30, 2019 at 6:30 AM shalom sagges wrote: > Thanks for your replies guys.

Re: Collecting Latency Metrics

2019-05-30 Thread Chris Lohfink
> > org.apache.cassandra.metrics.ClientRequest.Latency.Read these measure the > latency in milliseconds > Its actually in microseconds, unless calling the values() operation which gives the histogram in nanoseconds On Wed, May 29, 2019 at 4:34 PM Paul Chandler wrote: > There are various attribu

Re: Collecting Latency Metrics

2019-05-29 Thread Chris Lohfink
could end up with really bad queries that don't even show up as a tick on your graph (although *generally* it will). Chris On Wed, May 29, 2019 at 9:32 AM shalom sagges wrote: > Hi All, > > I'm creating a dashboard that should collect read/write latency metrics on > C* 3.x. &g

Re: GraalVM

2019-05-09 Thread Chris Hane
Awesome. Will try to join. Thanks for the links. Will look through them also. On Thu, May 9, 2019 at 8:33 AM Sebastian Estevez < sebastian.este...@datastax.com> wrote: > Hi Chris, > > Funny you mention this today of all days because we're doing a twitch > streaming se

GraalVM

2019-05-08 Thread Chris Hane
Has anyone worked with graalvm to include a cql driver in the native-image build? Looking to see if it is possible or known to not be possible? Thanks, Chris

Re: Cassandra config in table

2019-02-25 Thread Chris Lohfink
In 4.0+ you can SELECT * FROM system_views.settings; Chris On Mon, Feb 25, 2019 at 9:22 AM Abdul Patel wrote: > Do we have any sustem table which stores all config details which we have > in yaml or cassandra env.sh?

Re: Cassandra collection tombstones

2019-01-25 Thread Chris Lohfink
you have to provide a -g or --gc-grace-seconds parameter. I am not sure where the "always wrong" comes in as the quantity of data thats being shadowed is not what its tracking (although it would be more meaningful for single sstable compactions if it did), just when tombstones can be pu

Re: Compact storage removal effect

2019-01-22 Thread Chris Lohfink
In 3.x+ the format on disk is the same with compact storage on or off so you shouldn't expect much of a difference in table size with the new storage format compared to compact vs non compact in 2.x. Chris > On Jan 22, 2019, at 10:21 AM, Nitan Kainth wrote: > > hey Chris, &g

Re: Compact storage removal effect

2019-01-22 Thread Chris Lohfink
What version are you running? Did you include an upgradesstables -a or something to rebuild without the compact storage in your migration? After 3.0 the new format can be more or less the same size as the 2.x compact storage tables depending on schema (which can impact things a lot). Chris

How can I limit the non-heap memory for Cassandra

2019-01-02 Thread Chris Mildebrandt
the oom killer, so just placing limits on the container won't help. Thanks, -Chris

Re: High CPU usage on some of the nodes due to message coalesce

2018-10-20 Thread Chris Lohfink
repair coordination, wide partition reads, or compactions so need to look more at what within the app is causing the pressure to know if its possible to improve with settings or if the load your application is producing exceeds what your cluster can handle (needs more nodes). Chris > On Oct 20,

Re: jmxterm "#NullPointerException: No such PID "

2018-09-20 Thread Chris Lohfink
if its ever a significant portion of your GC pauses. Chris On Thu, Sep 20, 2018 at 6:05 AM Philip Ó Condúin wrote: > Thank you Yuki, this explains it. > I am used to working on C* 2.1 in production where this JVM flag is not > enabled. > > > On Wed, 19 Sep 2018 at 00:29, Yuk

Re: Setting up rerouting java/python driver read requests from unresponsive nodes to good ones

2018-08-15 Thread Chris Lohfink
/speculative_execution/ Chris Sent from my iPhone > On Aug 15, 2018, at 6:57 AM, Horia Mocioi wrote: > > Hello, > > I believe that this is what you are looking for - > https://docs.datastax.com/en/developer/java-driver/3.5/manual/retries/ > > In particular, tryNextHost(

Re: Cassandra Compaction Metrics - CompletedTasks vs TotalCompactionCompleted

2018-08-10 Thread Chris Lohfink
If its occurring that often you can monitor nodetool compactionstats to see whats running > On Aug 10, 2018, at 11:35 AM, Dionne Cloudoupoulos > wrote: > > On 2017/10/31 16:56:29, Chris Lohfink wrote: >> The "CompletedTasks" metric is a measure of how many tasks r

Re: concurrent_compactors via JMX

2018-07-18 Thread Chris Lohfink
6 org.apache.cassandra.db:type=CompactionManager,name=CoreCompactorThreads -> 6 Would set it to 6. To decrease them you will want to go opposite order (core than max). Just increasing the number of concurrent compactors doesnt mean that all of them will be utilized though. Chris > On Jul 17,

Re: Compaction process stuck

2018-07-05 Thread Chris Lohfink
` or restarting the node. Chris > On Jul 5, 2018, at 12:08 AM, atul atri wrote: > > Hi, > > We noticed that compaction process is also hanging on a node in backup ring. > Please find attached thread dump for both servers. Recently, we have made few > changes in cluster to

Re: Compaction process stuck

2018-07-04 Thread Chris Lohfink
Can you take a thread dump (jstack) and share the state of the compaction threads? Also check for “Exception” in logs Chris Sent from my iPhone > On Jul 4, 2018, at 8:37 AM, atul atri wrote: > > Hi, > > On one of our server, compaction process is hanging. It's stuck at

Re: G1GC CPU Spike

2018-06-15 Thread Chris Lohfink
There are no bad GCs in the gclog (worse is like 100ms). Everything looks great actually from what I see. CPU utilization isn't inherently a bad thing for what its worth. Chris > On Jun 14, 2018, at 1:18 PM, rajpal reddy wrote: > > Hey Chris, > > Sorry to bother you. Did

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
There are not even a 100ms GC pause in that, are you certain theres a problem? > On Jun 13, 2018, at 3:00 PM, rajpal reddy wrote: > > Thanks Chris I did attached the gc logs already. reattaching them > now. > > it started yesterday around 11:54PM >> On Jun 13, 2018, a

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
se yes it would be bad to set Xmn. Giving the gc logs will give the results of all the bash scripts along with details of whats happening so its your best option if you want help to share that. Chris > On Jun 13, 2018, at 12:17 PM, Subroto Barua > wrote: > > Chris, > Wha

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
That metric is the total number of seconds spent in GC, it will increase over time with every young gc which is expected. Whats interesting is the rate of growth not the fact that its increasing. If graphing tool has option to graph derivative you should use that instead. Chris > On Jun

Re: G1GC CPU Spike

2018-06-13 Thread Chris Lohfink
failures. Reserving more of the heap to be free (-XX:G1ReservePercent=25) can help, along with increasing the amount of heap. 8GB is pretty small for G1, might be better off with CMS. Chris > On Jun 13, 2018, at 8:42 AM, rajpal reddy wrote: > > Hello, > > we are using G1GC and n

Re: nodetool (2.1.18) - Xmx, ParallelGCThreads, High CPU usage

2018-05-29 Thread Chris Lohfink
capped too (along with parallel and concurrent gc threads). Chris Sent from my iPhone > On May 29, 2018, at 4:42 PM, kurt greaves wrote: > > Good to know. So that confirms it's just the GC threads causing problems. > >> On Tue., 29 May 2018, 22:02 Steinmaurer, Thom

"Group by" while limiting a clustering column with a range

2018-04-10 Thread Chris Mildebrandt
Hey all. I’m trying to use a range to limit a clustering column while at the same time using `group by` and running into issues. Here’s a sample table: create table if not exists samples (name text, partition int, sample int, city text, state text, count counter, primary key ((name, partition), sam

Re: tablestats and gossip

2018-04-06 Thread Chris Lohfink
Yes, its the count of all locally applied writes to that table. A insert to a table with a RF=3 should increase the local write count by 1 on 3 different nodes. Chris > On Apr 6, 2018, at 5:00 AM, Grzegorz Pietrusza wrote: > > Hi all > > Does local write count provided by tab

Re: Understanding Blocked and All Time Blocked columns in tpstats

2018-03-23 Thread Chris Lohfink
options to help with that (ie prepared queries, what queries are, limiting number of async inflight queries) Chris > On Mar 23, 2018, at 11:42 AM, John Sanda wrote: > > Thanks for the explanation. In the past when I have run into problems related > to CASSANDRA-11363, I have i

Re: Understanding Blocked and All Time Blocked columns in tpstats

2018-03-23 Thread Chris Lohfink
throw away on rejections). Worth noting this is only really possible in the native transport pool (sep pool) last I checked. Since 2.1 at least, before that there were a few others. That changes version to version. For (basically) all other thread pools the queue is limited by memory. Chris >

Re: Delete System_Traces Table

2018-03-19 Thread Chris Lohfink
selves but it doesnt hurt anything to have the table there. Just ignore it and its existence will not cause any issues. Chris > On Mar 19, 2018, at 10:27 AM, shalom sagges wrote: > > That's weird... I'm using 3.0.12, so I should've still been able to drop it, > no?

Re: Delete System_Traces Table

2018-03-19 Thread Chris Lohfink
lowing create. Chris > On Mar 19, 2018, at 9:15 AM, shalom sagges wrote: > > Yes, that's correct. > > I'd definitely like to keep the default tables. > > On Mon, Mar 19, 2018 at 4:10 PM, Rahul Singh <mailto:rahul.xavier.si...@gmail.com>> wrote: > I thi

Re: Delete System_Traces Table

2018-03-19 Thread Chris Lohfink
No. Why do you want to? If you don't use tracing they will be empty, and if were able to drop them you will no longer be able to use tracing in debugging. Chris > On Mar 19, 2018, at 7:52 AM, shalom sagges wrote: > > Hi All, > > I accidentally created a test table

Re: WARN [PERIODIC-COMMIT-LOG-SYNCER] .. exceeded the configured commit interval by an average of...

2018-03-16 Thread Chris Lohfink
If you just want to make it work, increase commitlog_segment_size_in_mb to 64. A single mutation cannot exceed 1/2 the segment size. If you want to actually fix your problem decrease the size of the mutations and limit the size of the value blob. <== recommended Chris > On Mar 16, 2018

Re: system.size_estimates - safe to remove sstables?

2018-03-06 Thread Chris Lohfink
While its off you can delete the files in the directory yeah Chris > On Mar 6, 2018, at 2:35 AM, Kunal Gangakhedkar > wrote: > > Hi Chris, > > I checked for snapshots and backups - none found. > Also, we're not using opscenter, hadoop or spark or any such tool. &g

Re: cfhistograms InstanceNotFoundException EstimatePartitionSizeHistogram

2018-03-06 Thread Chris Lohfink
. Chris Sent from my iPhone > On Mar 6, 2018, at 3:29 AM, onmstester onmstester wrote: > > Running this command: > nodetools cfhistograms keyspace1 table1 > > throws this exception in production server: > javax.management.InstanceNotFoundException: > org.apache.cass

Re: system.size_estimates - safe to remove sstables?

2018-03-05 Thread Chris Lohfink
Any chance space used by snapshots? What files exist there that are taking up space? > On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar > wrote: > > Hi all, > > I have a 2-node cluster running cassandra 2.1.18. > One of the nodes has run out of disk space and died - almost all of it shows > up

Re: system.size_estimates - safe to remove sstables?

2018-03-05 Thread Chris Lohfink
refreshsizeestimates` to readd it or just wait for it to re-run automatically (every 5 min). Chris > On Mar 5, 2018, at 1:02 AM, Kunal Gangakhedkar > wrote: > > Hi all, > > I have a 2-node cluster running cassandra 2.1.18. > One of the nodes has run out of disk space and died - a

Re: Cassandra Needs to Grow Up by Version Five!

2018-02-21 Thread Chris Lohfink
on it then trying to shame the community with a sales pitch from another DB's sales guy. Chris > On Feb 21, 2018, at 4:53 PM, Kenneth Brotman > wrote: > > Hi Akash, > > I get the part about outside work which is why in replying to Jeff Jirsa I > was suggesting the

Re: Commitlogs are filling the Full Disk space and nodes are down

2018-01-30 Thread Chris Lohfink
permission error (some tooling creates commitlogs so if run by wrong user can create this prooblem), or a memtable flush error. You can also check tpstats to see if tasks are queued up in postmemtable flusher and jstack to see where the active ones are stuck if they are. Chris > On Jan 30, 2018,

Re: sstabledump tries to delete a file

2018-01-10 Thread Chris Lohfink
Yes it should be read only, open a jira please. It does look like if the fp changed it would rebuild or if your missing. When it builds the table metadata from the sstable it can just set the properties to match that of the sstable to prevent this. Chris On Wed, Jan 10, 2018 at 4:16 AM

Re: sstable

2017-12-20 Thread Chris Lohfink
Somewhere along the line sstabledump tool incorrectly got setup to use tool initialization, its fixed https://issues.apache.org/jira/browse/CASSANDRA-13683 Chris On Tue, Dec 19, 2017 at 5:45 PM, Mounika kale wrote: > Hi, > I'm getting below error for all sstable tools. > &g

Re: gc causes C* node hang

2017-11-30 Thread Chris Lohfink
Mail client may be changing changing the char if your copy and pasting, its - "hyphen" not the unicode en dash –. I would recommend adding it to jvm options like oleksandr pointed out Chris On Thu, Nov 30, 2017 at 1:50 AM, Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote

Re: What is OneMinuteRate in Write Latency?

2017-11-03 Thread Chris Lohfink
erage> of events. Chris On Thu, Nov 2, 2017 at 12:10 PM, AI Rumman wrote: > Hi, > > I am trying to calculate the Read/second and Write/Second in my Cassandra > 2.1 cluster. After searching and reading, I came to know about JMX bean > "org.apache.cassandra.metrics:type=ClientReq

Re: Cassandra Compaction Metrics - CompletedTasks vs TotalCompactionCompleted

2017-10-31 Thread Chris Lohfink
CompactionMetrics is a combination of the compaction executor (sstable compactions, secondary index build, view building, relocate, garbagecollect, cleanup, scrub etc) and validation executor (repairs). Keep in mind not all jobs execute 1 task per operation, things that use the parallelAllSSTableOp

Re: Inter Data Center Latency calculation of a Multi DC cluster running in AWS

2017-10-17 Thread Chris Lohfink
ons) take to get to a node from other DCs. Chris On Tue, Oct 17, 2017 at 7:18 PM, Jon Haddad wrote: > I recommend figuring out the latency between your datacenters. Cassandra > isn’t going to be any more than that barring JVM pauses on the remote > coordinator. > > > On Oct

Re: Cassandra and G1 Garbage collector stop the world event (STW)

2017-10-09 Thread Chris Lohfink
Can you share your schema and cfstats? This sounds kinda like a wide partition, backed up compactions, or tombstone issue for it to create so much and have issues like that so quickly with those settings. A heap dump would be most telling but they are rather large and hard to share. Chris On

Re: [EXTERNAL] Re: Increasing VNodes

2017-10-04 Thread Chris Lohfink
earn about how > reaper works: http://cassandra-reaper.io/ > <https://urldefense.proofpoint.com/v2/url?u=http-3A__cassandra-2Dreaper.io_&d=DwMFAg&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=O20_rcIS1QazTO3_J10I1cPIygxnuBZ4sUCz1TS16XE&m=nHN7toaSQUjfwSABx1KX

Re: Increasing VNodes

2017-10-04 Thread Chris Lohfink
://github.com/thelastpickle/cassandra-reaper or DataStax OpsCenter's repair service is easiest solution without a lot of effort. Repairs are hard. Chris On Wed, Oct 4, 2017 at 11:48 AM, Jeff Jirsa wrote: > You don't need to change the number of vnodes, you can manually select >

Re: Read-/ Write Latency - Cassandra 2.1 .15 vs 3.10

2017-10-03 Thread Chris Lohfink
light=metrics#table-metrics <http://cassandra.apache.org/doc/latest/operating/metrics.html?highlight=metrics#table-metrics> Chris > On Oct 3, 2017, at 10:06 AM, Anumod Mullachery > wrote: > > Hi, We were running splunk queries to pull read / write latency. It's > w

Re: Do not use Cassandra 3.11.0+ or Cassandra 3.0.12+

2017-09-12 Thread Chris Lohfink
Last Ive seen of it OpsCenter does not collect this metric. I don't think any monitoring tools do. Chris > On Sep 11, 2017, at 4:06 PM, CPC wrote: > > Hi, > > Is this bug fixed in dse 5.1.3? As I understand calling jmx getTombStoneRatio > trigers that bug. We are usin

Re: Cassandra CF Level Metrics (Read, Write Count and Latency)

2017-09-01 Thread Chris Lohfink
queries) mbean which may be what your looking for. Table level write coordinator metrics are missing since the read coordinator metrics were actually added for speculative retry so I think writes were overlooked. Chris On Thu, Aug 31, 2017 at 10:58 PM, Jai Bheemsen Rao Dhanwada < jaibheem...@gmail

Re: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory

2017-08-31 Thread Chris Lohfink
ffers at startup but does I/O with small buffers may see a > benefit to using this property. Applications that do I/O using direct > buffers will not see any benefit to using this system property. > See JDK-8147468 <http://bugs.openjdk.java.net/browse/JDK-8147468> Chris On Thu, A

Re: Nodetool tablehistograms

2017-07-19 Thread Chris Lohfink
Its the number of sstables that may of been read from. This includes sstables who had their bloom filters checked (which may hit disk). This changes a bit in https://issues.apache.org/jira/browse/CASSANDRA-13120 to be only the sstables that its actually reading from. On Wed, Jul 19, 2017 at 11:04

Re: reduced num_token = improved performance ??

2017-07-12 Thread Chris Lohfink
compaction. I would highly recommend using far less than 256 in 3.x. Chris On Tue, Jul 11, 2017 at 8:36 PM, Justin Cameron wrote: > Hi, > > Using fewer vnodes means you'll have a higher chance of hot spots in your > cluster. Hot spots in Cassandra are nodes that, by random chance,

Re: Understanding of cassandra metrics

2017-07-07 Thread Chris Lohfink
which ViewLockAcquireTime gives visibility too. Also there are sometimes reads required for updating materialized views, which ViewReadTime is for tracking. For more details id recommend https://opencredo.com/everything-need-know-cassandra-materialized-views/ Chris On Fri, Jul 7, 2017 at 9:42 AM

Re: Partition range incremental repairs

2017-06-19 Thread Chris Stokesmore
Anyone have anymore thoughts on this at all? Struggling to understand it.. > On 9 Jun 2017, at 11:32, Chris Stokesmore > wrote: > > Hi Anuj, > > Thanks for the reply. > > 1). We are using Cassandra 2.2.8, and our repair commands we are comparing > are >

Re: Partition range incremental repairs

2017-06-09 Thread Chris Stokesmore
ugs!? I guess massive over streaming while a performance issue, does not affect data integrity.. Are there any plans to back port this to 3 or ideally 2.2 ? Chris > On Tue, Jun 6, 2017 at 9:54 AM Anuj Wadehra > wrote: > Hi Chris, > > Can your share following info: > > 1.

Re: Partition range incremental repairs

2017-06-09 Thread Chris Stokesmore
currently running repair has completed. Many thanks for the reply again, Chris > On 6 Jun 2017, at 17:50, Anuj Wadehra wrote: > > Hi Chris, > > Can your share following info: > > 1. Exact repair commands you use for inc repair and pr repair > > 2. Repair time sh

Re: Partition range incremental repairs

2017-06-06 Thread Chris Stokesmore
up as all three sets of data on each repair job should be marked as repaired however this does not seem to be the case. Any ideas? Chris > On 6 Jun 2017, at 16:08, Anuj Wadehra wrote: > > Hi Chris, > > Using pr with incremental repairs does not make sense. Primary range

Re: Partition range incremental repairs

2017-06-06 Thread Chris Stokesmore
Hi all, Wondering if anyone had any thoughts on this? At the moment the long running repairs cause us to be running them on two nodes at once for a bit of time, which obivould increases the cluster load. On 2017-05-25 16:18 (+0100), Chris Stokesmore wrote: > Hi,> > > We are

Partition range incremental repairs

2017-05-25 Thread Chris Stokesmore
Hi, We are running a 7 node Cassandra 2.2.8 cluster, RF=3, and had been running repairs with the —pr option, via a cron job that runs on each node once per week. We changed that as some advice on the Cassandra IRC channel said it would cause more anticompaction and http://docs.datastax.com/e

Re: what is MemtableReclaimMemory mean ??

2017-05-01 Thread Chris Lohfink
Question though, how many tables do you have? If you have more than a few hundreds it could be bottlenecking the flushing if it is flushing very frequently. On Mon, May 1, 2017 at 9:32 PM, Chris Lohfink wrote: > Theres a read barrier to stop reclaiming a memtable when there are > re

Re: what is MemtableReclaimMemory mean ??

2017-05-01 Thread Chris Lohfink
resources which might cause additional heap allocation pressure. Its more likely a symptom of GCs or reads being slow than the cause of the issue however. Chris On Mon, May 1, 2017 at 9:01 PM, Pranay akula wrote: > Hi Alain, > > when "*MemtableReclaimMemory*" Pending Tasks inc

Re: How can I efficiently export the content of my table to KAFKA

2017-04-27 Thread Chris Stromberger
Maybe https://www.confluent.io/blog/kafka-connect-cassandra-sink-the-perfect-match/ On Wed, Apr 26, 2017 at 2:49 PM, Tobias Eriksson < tobias.eriks...@qvantel.com> wrote: > Hi > > I would like to make a dump of the database, in JSON format, to KAFKA > > The database contains lots of data, milli

Re: Node always dieing

2017-04-10 Thread Chris Mawata
Notice .SimpleSeedProvider{seeds=10.100.100.19, 10.100.100.85, 10.100.100.185, 10.100.100.161, 10.100.100.52, 10.100.1000.213}; Why do you have all six of your nodes as seeds? is it possible that the last one you added used itself as the seed and is isolated? On Thu, Apr 6, 2017 at 6:48 AM, Cogum

Re: system_auth replication strategy

2017-04-01 Thread Chris Lohfink
[ ] [*] [*] [*] [ ] In which case an insert or read to a node on DC1 with LOCAL_ONE or LOCAL_QUORUM will result in an unavailable exception. Chris > On Apr 1, 2017, at 10:51 AM, Vlad wrote: > > Hi, > > what is the suitable replication strategy for system_auth keyspace? &g

Re: nodes are always out of sync

2017-04-01 Thread Chris Lohfink
perfectly in sync for a repair. I wouldnt worry about that log message. If you are worried about consistency between your read/writes use each or local quorum for both. Chris On Thu, Mar 30, 2017 at 1:22 AM, Roland Otta wrote: > hi, > > we see the following behaviour in our environment: &g

Re: partition sizes reported by nodetool tablehistograms

2017-02-24 Thread Chris Lohfink
Its the decompressed size of the partitions. Each sstable has stats component that contains histograms for the size and number of columns in the partitions (among other things, can see with sstablemetadata tool), tablehistograms merges it for each sstable and gives the results. Chris On Fri, Feb

Re: Backups eating up disk space

2017-01-15 Thread Chris Mawata
You don't have a viable solution because you are not making a snapshot as a starting point. After a while you will have a lot of backup data. Using the backups to get your cluster to a given state will involve copying a very large amount of backup data, possibility more than the capacity of your c

Re: Help

2017-01-09 Thread Chris Lohfink
Do you have any monitoring setup around garbage collections? A GC + network latency > write timeout will cause intermittent hints. On Sun, Jan 8, 2017 at 10:30 PM, Anshu Vajpayee wrote: > Gossip shows - all nodes are up. > > But when we perform writes , coordinator stores the hints. It means

Re: Java GC pauses, reality check

2016-11-25 Thread Chris Lohfink
tombstones, very wide partitions). Chris On Fri, Nov 25, 2016 at 3:25 PM, S Ahmed wrote: > Hello! > > From what I understand java GC pauses are pretty much a fact of life, but > you can tune the jvm to reduce the likelihood of the frequency and length > of GC pauses. > > When

Re: Can a Select Count(*) Affect Writes in Cassandra?

2016-11-10 Thread Chris Lohfink
"pending task" metric is the measure of how many mutations are blocked by this lock). Chris On Thu, Nov 10, 2016 at 8:10 AM, Shalom Sagges wrote: > Hi Alexander, > > I'm referring to Writes Count generated from JMX: > [image: Inline image 1] > > The higher cu

Re: metrics not resetting after running proxyhistograms or cfhistograms

2016-10-25 Thread Chris Lohfink
That behavior went away with 2.2. https://issues.apache.org/jira/browse/CASSANDRA-11752 adds decay to it to make it recent data which is much better then just reseting on reads. Chris On Tue, Oct 25, 2016 at 2:06 PM, Andrew Bialecki < andrew.biale...@klaviyo.com> wrote: > We'

Re: system_distributed.repair_history table

2016-10-06 Thread Chris Lohfink
lly truncate as it is > only for debugging purposes > > Naidu Saladi > > > On Wednesday, October 5, 2016 8:03 PM, Chris Lohfink > wrote: > > > The only current solution is to truncate it periodically. I opened > https://issues.apache.org/jira/browse/CASSANDRA-12701 a

Re: system_distributed.repair_history table

2016-10-05 Thread Chris Lohfink
The only current solution is to truncate it periodically. I opened https://issues.apache.org/jira/browse/CASSANDRA-12701 about it if interested in following On Wed, Oct 5, 2016 at 4:23 PM, Saladi Naidu wrote: > We are seeing following warnings in system.log, As > *compaction_large_partition_war

Re: repair_history maintenance

2016-09-23 Thread Chris Lohfink
Probably should just periodically truncate/clear snapshots when gets too big (will probably take months before noticeable). I opened https://issues.apache.org/jira/browse/CASSANDRA-12701 for discussion on if it should use TTLs Chris On Thu, Sep 22, 2016 at 1:28 PM, sfesc...@gmail.com wrote

Finding records that exist on Cassandra but not externally

2016-09-07 Thread chris
7;ll just be overwriting existing keys. I'd be grateful if anyone could shed some advice on the best solution here or whether there's some better way I haven't thought of. Thanks, Chris

Re: How to get information of each read/write request?

2016-08-30 Thread Chris Lohfink
Running a query with trace (`TRACING ON` in cqlsh) can give you a lot of the information for an individual request. There has been a ticket to track time in queue (https://issues.apache.org/jira/browse/CASSANDRA-8398) but no ones worked on it yet. Chris On Tue, Aug 30, 2016 at 12:20 PM, Jun Wu

Re: Support/Consulting companies

2016-08-19 Thread Chris Tozer
at provide > contracting services related to Cassandra. Is there a comprehensive list > somewhere? Alternatively could you folks tell me who you use? > > Thanks in advance for any replies! > > Roxy > > -- Chris Tozer Instaclustr (408) 781-7914 Spin Up a Free 14 Day Trial <https://www.instaclustr.com>

Re: Hintedhandoff mutation

2016-08-17 Thread Chris Lohfink
Probably question better suited for the dev@ list. But I afaik the answer is there is no way to tell the difference, but probably safe to look at the created time, HHs tend to be older. Chris On Wed, Aug 17, 2016 at 5:02 AM, Stone Fang wrote: > Hi All, > > I want to differ hint

Re: a solution of getting cassandra cross-datacenter latency at a certain time

2016-08-08 Thread Chris Lohfink
bins range during the period. Also can wait for CASSANDRA-11752 <https://issues.apache.org/jira/browse/CASSANDRA-11752> for the a "recent" histogram (although would need to apply it to this histogram as well). Chris Lohfink On Mon, Aug 8, 2016 at 8:50 AM, Ryan Svihla wrote: >

Re: Approximate row count

2016-07-27 Thread Chris Lohfink
the number of keys are the number of *partition keys, *not row keys. You have ~39434 partitions, ranging from 311 bytes to 386mb. Looks like you have some wide partitions that contain many of your rows. Chris Lohfink On Wed, Jul 27, 2016 at 1:44 PM, Luke Jolly wrote: > I have a table that

RE: My cluster shows high system load without any apparent reason

2016-07-22 Thread Chris Lee
Unsubscribe me. Thank you From: Ryan Svihla [mailto:r...@foundev.pro] Sent: viernes, 22 de julio de 2016 14:39 To: user@cassandra.apache.org Subject: Re: My cluster shows high system load without any apparent reason You aren't using counters by chance? regards, Ryan Svihla On Jul 22, 2016, 2:

Re: sstabledump failing for system keyspace tables

2016-06-11 Thread Chris Lohfink
ake advantage of the issue mentioned in this comment <https://issues.apache.org/jira/browse/CASSANDRA-11330?focusedCommentId=15226927&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15226927> Chris On Sat, Jun 11, 2016 at 3:00 PM, Bhuvan Rawal wrote: >

Re: Latency overhead on Cassandra cluster deployed on multiple AZs (AWS)

2016-04-11 Thread Chris Lohfink
n-different-EC2-availability-zones-within-the-same-region Are you using EBS? That would further impact latency on reads and GCs will always cause hiccups in the 99th+. Chris On Mon, Apr 11, 2016 at 7:57 AM, Alessandro Pieri wrote: > Hi everyone, > > Last week I ran some tests to estim

Re: Datastax OpsCenter - Can't connect to Cassandra All host(s) tried for query failed

2016-04-10 Thread Chris Lohfink
On the Cassandra nodes, does the output from `netstat -anp | grep 9042` match the 10.5.0.20x interface? Are there any firewalls/NATs in place? You can try to connect from the opscenter node to the 10.5.0.201 node via cqlsh, and try to work from there. Chris On Sun, Apr 10, 2016 at 5:18 AM, okan

Cassandra nodes using internal network to try and talk externally

2016-04-07 Thread Chris Elsmore
a loss! Chris

Re: DataModelling to query date range

2016-03-24 Thread Chris Martin
Ah- that looks interesting! I'm actaully still on cassandra 2.x but I was planning on updgrading anyway. Once I do so I'll check this one out. Chris On Thu, Mar 24, 2016 at 2:57 AM, Henry M wrote: > I haven't tried the new SASI indexer but it may help: > https://github

  1   2   3   4   5   >