It did, but a ran it again on one node – that node never recovered. ☹
From: Robert Coli [mailto:rc...@eventbrite.com]
Sent: 02 October 2015 21:20
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
On Fri, Oct 2, 2015 at 1:32 AM, Walsh, Stephen
mailto:stephen.wa...@aspect.com>>
On Fri, Oct 2, 2015 at 1:32 AM, Walsh, Stephen
wrote:
> Sorry for the late reply, I ran the nodetool resetlocalschema on all
> nodes but in the end it just removed all the schemas and crashed the
> applications.
>
> I need to reset and try again. I’ll try get you the gc stats today J
>
FTR, runn
en.wa...@aspect.com]
Sent: 02 October 2015 09:32
To: user@cassandra.apache.org
Subject: RE: Consistency Issues
Sorry for the late reply, I ran the nodetool resetlocalschema on all nodes but
in the end it just removed all the schemas and crashed the applications.
I need to reset and try again. I’ll try ge
:01
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
You say that you don't think GC is your issue... but did you actually check?
The reasons you suggest aren't very convincing. Can you provide your GC
settings, and take a look at jstat --gccause?
http://docs.oracle.co
t;
>
>
> But I really need to understand this best practice that was mentioned (on
> number of CF’s) by Jack Krupansky.
>
> Anyone more information on this?
>
>
>
>
>
> Many thanks for all your help guys keep it coming J
>
> Steve
>
>
>
> *From:*
k...@gmail.com]
> *Sent:* 01 October 2015 14:00
> *To:* user
> *Subject:* Re: Consistency Issues
>
>
>
> Onur, was responding to Stephen's issue.
>
>
>
>
>
> On Thu, Oct 1, 2015 at 8:56 AM, Onur Yalazı
> wrote:
>
> Thank you Jake.
>
>
ode 1 : has all CF’s
>
> Node 2 : has all CF’s
>
> Node 3 : has all CF’s
>
> Node 4 : has all CF’s
>
>
>
>
>
> This is indeed very strange….
>
>
>
>
>
> *From:* Carlos Alonso [mailto:i...@mrcalonso.com]
> *Sent:* 01 October 2015 12:05
> *To:*
Thanks Jake, I’ll try test out 2.1.9 to see if it resolved the issue and ill
try “nodetool resetlocalschema” now to see if it helps.
Cassandra is 2.1.6
OpsCenter is 5.2.1
From: Jake Luciani [mailto:jak...@gmail.com]
Sent: 01 October 2015 14:00
To: user
Subject: Re: Consistency Issues
Onur
Onur, was responding to Stephen's issue.
On Thu, Oct 1, 2015 at 8:56 AM, Onur Yalazı wrote:
> Thank you Jake.
>
> The issue is I do not have missing CF's and upgrading beyond 2.1.3 is not
> a possibility because of the deprecation of cql dialects. Our application
> is using Hector and migrating
Thank you Jake.
The issue is I do not have missing CF's and upgrading beyond 2.1.3 is
not a possibility because of the deprecation of cql dialects. Our
application is using Hector and migrating to cql3 is a huge refactoring.
On 01/10/15 15:48, Jake Luciani wrote:
Couple things to try.
1. n
Couple things to try.
1. nodetool resetlocalschema on the nodes with missing CFs. This will
refresh the schema on the local node.
2. upgrade to 2.1.9. There are some pretty major issues in 2.1.6 (nothing
specific to this problem but worth upgrading)
deed very strange….
>
>
>
>
>
> *From:* Carlos Alonso [mailto:i...@mrcalonso.com]
> *Sent:* 01 October 2015 12:05
> *To:* user@cassandra.apache.org
> *Subject:* Re: Consistency Issues
>
>
>
> And that's a stupid one, I know, but does the column you're tryi
CF’s
This is indeed very strange….
From: Carlos Alonso [mailto:i...@mrcalonso.com]
Sent: 01 October 2015 12:05
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
And that's a stupid one, I know, but does the column you're trying to access
actually exist?
Carlos Alonso | Sof
J
>
>
>
>
>
> *From:* Carlos Alonso [mailto:i...@mrcalonso.com]
> *Sent:* 01 October 2015 10:11
>
> *To:* user@cassandra.apache.org
> *Subject:* Re: Consistency Issues
>
>
>
> Hi Stephen.
>
>
>
> The UnknownColumnFamilyException made me tho
I did think of that and they are all the same version ☺
From: Carlos Alonso [mailto:i...@mrcalonso.com]
Sent: 01 October 2015 10:11
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
Hi Stephen.
The UnknownColumnFamilyException made me thought of a possible schema
disagreement in
help guys keep it coming J
>
> Steve
>
>
>
> *From:* Ricardo Sancho [mailto:sancho.rica...@gmail.com]
> *Sent:* 01 October 2015 09:39
> *To:* user@cassandra.apache.org
> *Subject:* RE: Consistency Issues
>
>
>
> Can you tell us how much time your gcs are taking?
ancho [mailto:sancho.rica...@gmail.com]
Sent: 01 October 2015 09:39
To: user@cassandra.apache.org
Subject: RE: Consistency Issues
Can you tell us how much time your gcs are taking?
Do you see any especially long ones?
On 1 Oct 2015 09:37, "Walsh, Stephen"
mailto:stephen.wa...@aspect.com>> wr
ot the root causing of the
> inconsistency issue.
>
>
>
> Can anyone verify the best practice for number of CF’s?
>
>
>
>
>
> *From:* Robert Coli [mailto:rc...@eventbrite.com]
> *Sent:* 30 September 2015 18:45
> *To:* user@cassandra.apache.org
> *Subject:
September 2015 18:45
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
On Wed, Sep 30, 2015 at 9:06 AM, Walsh, Stephen
mailto:stephen.wa...@aspect.com>> wrote:
We never had these issue with our first run. Its only when we added another 25%
of writes.
As Jack said, you are pr
On Wed, Sep 30, 2015 at 9:06 AM, Walsh, Stephen
wrote:
>
> We never had these issue with our first run. Its only when we added
> another 25% of writes.
>
As Jack said, you are probably pushing your GC over a threshold, leading to
long pause times and inability to meet quorum.
As Sebastian said,
t in best practices for the number of CF’s?
From: Sebastian Estevez [mailto:sebastian.este...@datastax.com]
Sent: 30 September 2015 17:29
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
Can you provide exact details on where your load balancer is? Like Michael
said, you shouldn&#
.
>
>
>
> Many thanks for taking the time to reply Jack
>
>
>
>
>
>
>
> *From:* Jack Krupansky [mailto:jack.krupan...@gmail.com]
> *Sent:* 30 September 2015 16:53
> *To:* user@cassandra.apache.org
> *Subject:* Re: Consistency Issues
>
>
>
> M
these issue with our first run. Its only when we added another 25%
of writes.
Many thanks for taking the time to reply Jack
From: Jack Krupansky [mailto:jack.krupan...@gmail.com]
Sent: 30 September 2015 16:53
To: user@cassandra.apache.org
Subject: Re: Consistency Issues
More than "low hun
More than "low hundreds" (200 or 300 max, and preferably under 100) of
tables/column families is not exactly a recommended best practice. You may
be able to get it to work, but probably only with very heavy tuning (i.e.,
lots of time and playing with options) on your own part. IOW, no quick and
eas
What client are you using?
Official java and python clients should not have a LB between them and the
C* nodes AFAIK.
Why aren't you using 2.1.9?
Have you checked for schema agreement amongst all nodes?
ml
On Wed, Sep 30, 2015 at 11:22 AM, Walsh, Stephen
wrote:
> More information,
>
>
>
> I’
More information,
I've just setup a NTP server to rule out any timing issues.
And I also see this in the Cassandra node log files
MessagingService-Incoming-/172.31.22.4] 2015-09-30 15:19:14,769
IncomingTcpConnection.java:97 - UnknownColumnFamilyException reading from
socket; closing
org.apache.
It looks like NTP was the problem. Thanks for the solution!!!
On Wed, May 13, 2015 at 9:20 AM, Robert Wille wrote:
> Timestamps have millisecond granularity. If you make multiple writes
> within the same millisecond, then the outcome is not deterministic.
>
> Also, make sure you are running n
Timestamps have millisecond granularity. If you make multiple writes within the
same millisecond, then the outcome is not deterministic.
Also, make sure you are running ntp. Clock skew will manifest itself similarly.
On May 13, 2015, at 3:47 AM, Jared Rodriguez
mailto:jrodrig...@kitedesk.com>>
Thanks for the feedback. We have dug in deeper and upgraded to Cassandra
2.0.14 and are seeing the same issue. What appears to be happening is that
if a record is initially written, then the first read is fine. But if we
immediately update that record with a second write, that then the second
re
On Tue, May 12, 2015 at 12:35 PM, Michael Shuler
wrote:
> This is a 4 node cluster running Cassandra 2.0.6
>>
>
> Can you reproduce the same issue on 2.0.14? (or better yet, the
> cassandra-2.0 branch HEAD, which will soon ship 2.0.15) If you get the same
> results, please, open a JIRA with the r
On 05/12/2015 04:50 AM, Jared Rodriguez wrote:
I have a specific update and query that I need to ensure has strong
consistency. To that end, when I do the write, I set the consistency
level to ALL. Shortly afterwards, I do a query for that record with a
consistency of ONE and somehow get back s
31 matches
Mail list logo
