Re: Improve the performance of CAS

[Jason Brown]

Hey all,

Before we go bananas, let's see if Sylvain, the primary author of the
original patch, has the opportunity to chime with some explanatory notes or
other guidance. There may be some subtle points or considerations that are
not obvious, and I'd hate to lose that context.

Thanks,

-Jason

On Wed, May 16, 2018 at 2:57 PM, Ariel Weisberg  wrote:

> Hi,
>
> I think you are looking at the right low hanging fruit.  Cassandra
> deserves a better consensus protocol, but it's a very big project.
>
> Regards,
> Ariel
> On Wed, May 16, 2018, at 5:51 PM, Dikang Gu wrote:
> > Cool, create a jira for it,
> > https://issues.apache.org/jira/browse/CASSANDRA-14448. I have a draft
> patch
> > working internally, will clean it up.
> >
> > The EPaxos is more complicated, could be a long term effort.
> >
> > Thanks
> > Dikang.
> >
> > On Wed, May 16, 2018 at 2:20 PM, sankalp kohli 
> > wrote:
> >
> > > Hi,
> > > The idea of combining read with prepare sounds good. Regarding
> reducing
> > > the commit round trip, it is possible today by giving a lower
> consistency
> > > level for commit I think.
> > >
> > > Regarding EPaxos, it is a large change and will take longer to land. I
> > > think we should do this as it will help lower the latencies a lot.
> > >
> > > Thanks,
> > > Sankalp
> > >
> > > On Wed, May 16, 2018 at 2:15 PM, Jeremy Hanna <
> jeremy.hanna1...@gmail.com>
> > > wrote:
> > >
> > > > Hi Dikang,
> > > >
> > > > Have you seen Blake’s work on implementing egalitarian paxos or
> epaxos*?
> > > > That might be helpful for the discussion.
> > > >
> > > > Jeremy
> > > >
> > > > * https://issues.apache.org/jira/browse/CASSANDRA-6246
> > > >
> > > > > On May 16, 2018, at 3:37 PM, Dikang Gu  wrote:
> > > > >
> > > > > Hello C* developers,
> > > > >
> > > > > I'm working on some performance improvements of the lightweight
> > > > transitions
> > > > > (compare and set), I'd like to hear your thoughts about it.
> > > > >
> > > > > As you know, current CAS requires 4 round trips to finish, which
> is not
> > > > > efficient, especially in cross DC case.
> > > > > 1) Prepare
> > > > > 2) Quorum read current value
> > > > > 3) Propose new value
> > > > > 4) Commit
> > > > >
> > > > > I'm proposing the following improvements to reduce it to 2 round
> trips,
> > > > > which is:
> > > > > 1) Combine prepare and quorum read together, use only one round
> trip to
> > > > > decide the ballot and also piggyback the current value in response.
> > > > > 2) Propose new value, and then send out the commit request
> > > > asynchronously,
> > > > > so client will not wait for the ack of the commit. In case of
> commit
> > > > > failures, we should still have chance to retry/repair it through
> hints
> > > or
> > > > > following read/cas events.
> > > > >
> > > > > After the improvement, we should be able to finish the CAS
> operation
> > > > using
> > > > > 2 rounds trips. There can be following improvements as well, and
> this
> > > can
> > > > > be a start point.
> > > > >
> > > > > What do you think? Did I miss anything?
> > > > >
> > > > > Thanks
> > > > > Dikang
> > > >
> > > >
> > > > 
> -
> > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Dikang
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: 3.11.3

[Jason Brown]

wrt releasing 3.11, +1 to this ... and additionally, 3.0 and 2.2.

I'd propose we cut early next week (Monday) and anything that can get
reviewed/committed from Kurt's list would be great.

On Wed, Jun 6, 2018 at 5:22 AM, kurt greaves  wrote:

> We probably need to release 3.11.3 at some point soon because there's some
> pretty major bug fixes in there and interest has been expressed multiple
> times now for a release.
>
> Currently the only tickets marked for 3.11.3 that aren't complete are:
> https://issues.apache.org/jira/browse/CASSANDRA-14355 - No work has
> started
> on this yet.
> https://issues.apache.org/jira/browse/CASSANDRA-13929 - A lot of work has
> been done here... just need a final review
>
> However we've got quite a few tickets with patches that should probably be
> considered - can't say all are necessary to get into 3.11.3 but seeing as
> they all have patch available and have been around :
> for a while they are worth considering for anyone who could kindly review:
> https://issues.apache.org/jira/browse/CASSANDRA-14242 - Inconsistent
> indexing on static columns :(. needs review
> https://issues.apache.org/jira/browse/CASSANDRA-14415 - performance
> regression fix for DISTINCT
> https://issues.apache.org/jira/browse/CASSANDRA-14204 - Straightforward
> fix
> for nodetool garbagecollect
> https://issues.apache.org/jira/browse/CASSANDRA-14167 - Pretty much ready
> for commit - has been given +1 by Sylvain
> https://issues.apache.org/jira/browse/CASSANDRA-14099 - Needs only a unit
> test
> https://issues.apache.org/jira/browse/CASSANDRA-14085 - performance fix,
> small patch needs review
> https://issues.apache.org/jira/browse/CASSANDRA-13935 - Simple fix to CQL
> output for indexes when dumping schema
> https://issues.apache.org/jira/browse/CASSANDRA-13917 - Needs review, fix
> for compact storage inserts.
> https://issues.apache.org/jira/browse/CASSANDRA-13843 - Needs review.
> Small
> debian packaging fix for heapdump location.
> https://issues.apache.org/jira/browse/CASSANDRA-13834 - Needs an
> "official"
> reviewer and probably a test.
> https://issues.apache.org/jira/browse/CASSANDRA-13618 - Can likely be
> committed. Jeff has already done all the work for it just need to verify
> tests.
> https://issues.apache.org/jira/browse/CASSANDRA-11479 - Waiting for review
> for quite a long time.. unit test fix but seems to include changes to the
> server
>
> Handy JQL:
> project = CASSANDRA AND ((fixVersion = 3.11.x and status = "Patch
> Available" ) OR (fixVersion = 3.11.3 AND status != Resolved)) and Type =
> Bug
>

Re: 3.11 currently not building

[Jason Brown]

Thanks for the quick turn around, Lerh and Paulo!

On Thu, Jun 7, 2018 at 3:56 AM, Paulo Motta 
wrote:

> Fixed bad merge to 3.11 with 7bb88deb4c6387fd67114543986774c903860de9.
> Thanks for the heads up!
>
> 2018-06-07 2:34 GMT-03:00 Lerh Chuan Low :
> > Hi guys,
> >
> > I saw your message on IRC Jason, Kurt forwarded it to me. Thanks for the
> > heads up!
> >
> > 3.11 presently doesn't build, I think it may have been due to an
> accidental
> > merge gone bad. The original JIRA
> > https://issues.apache.org/jira/browse/CASSANDRA-13698 has been updated
> with
> > a patch and the tests are running (runs locally for me as well), do
> please
> > have a look when possible, thanks!
> >
> > Lerh
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

reroll the builds?

[Jason Brown]

Hey all,

The recent builds were -1'd, but it appears the issues have been resolved
(2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
CASSANDRA-14252). Can we go ahead and reroll now?

Thanks,

-Jason

Re: reroll the builds?

[Jason Brown]

Hi all,

First, thanks Joey for running the tests. Your pass/fail counts are
basically what in line with what I've seen for the last several months. (I
don't have an aggregated list anywhere, just observations from recent runs).

Second, it's beyond me why there's such inertia to actually cutting a
release. We're getting up to almost *six months* since the last release.
Are there any grand objections at this point?

Thanks,

-Jason


On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch  wrote:

> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
> various failing dtests but all three have green unit tests.
>
> 3.11.3 tentative (31d5d87, test branch
> <https://circleci.com/gh/vinaykumarchella/cassandra/
> tree/cassandra_3.11_temp_testing>,
> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258> pass,
> 5
> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
> <https://circleci.com/gh/vinaykumarchella/cassandra/256#tests/containers/8
> >
> dtest failures)
> 3.0.17 tentative (d52c7b8, test branch
> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
> unit
> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
> <https://circleci.com/gh/jolynch/cassandra/112> and 15
> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
> 2.2.13 tentative (3482370, test branch
> <https://circleci.com/gh/sumanth-pasupuleti/workflows/
> cassandra/tree/2.2-testing>,
> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
> pass, 9
> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/21> and 10
> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/
> 22#tests/containers/8>
> dtest failures)
>
> It looks like many (~6) of the failures in 3.0.x are related to
> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
>
> I don't see a good historical record to know if these are just flakes, but
> if we only want to go on green builds perhaps we can either disable the
> flakey tests or fix them up? If someone feels strongly we should fix
> particular tests up please link a jira and I can take a whack at some of
> them.
>
> -Joey
>
> On Tue, Jul 17, 2018 at 9:35 AM Michael Shuler 
> wrote:
>
> > On 07/16/2018 11:27 PM, Jason Brown wrote:
> > > Hey all,
> > >
> > > The recent builds were -1'd, but it appears the issues have been
> resolved
> > > (2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
> > > CASSANDRA-14252). Can we go ahead and reroll now?
> >
> > Could someone run through the tests on 2.2, 3.0, 3.11 branches and link
> > them?  Thanks!
> >
> > Michael
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: reroll the builds?

[Jason Brown]

I spoke with some people over here, and I'm going to spend a day doing a
quick triage of the failing dtests. There are some fixes for data loss bugs
that are critical to get out in these builds, so I'll ensure the current
failures are within an acceptable level of flakey-ness in order to unblock
those fixes.

Will have an update shortly ...

-Jason

On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown  wrote:

> Hi all,
>
> First, thanks Joey for running the tests. Your pass/fail counts are
> basically what in line with what I've seen for the last several months. (I
> don't have an aggregated list anywhere, just observations from recent runs).
>
> Second, it's beyond me why there's such inertia to actually cutting a
> release. We're getting up to almost *six months* since the last release.
> Are there any grand objections at this point?
>
> Thanks,
>
> -Jason
>
>
> On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> wrote:
>
>> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
>> various failing dtests but all three have green unit tests.
>>
>> 3.11.3 tentative (31d5d87, test branch
>> <https://circleci.com/gh/vinaykumarchella/cassandra/tree/
>> cassandra_3.11_temp_testing>,
>> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258>
>> pass, 5
>> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
>> <https://circleci.com/gh/vinaykumarchella/cassandra/256#
>> tests/containers/8>
>> dtest failures)
>> 3.0.17 tentative (d52c7b8, test branch
>> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
>> unit
>> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
>> <https://circleci.com/gh/jolynch/cassandra/112> and 15
>> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
>> 2.2.13 tentative (3482370, test branch
>> <https://circleci.com/gh/sumanth-pasupuleti/workflows/cassan
>> dra/tree/2.2-testing>,
>> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
>> pass, 9
>> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/21> and 10
>> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/22#
>> tests/containers/8>
>> dtest failures)
>>
>> It looks like many (~6) of the failures in 3.0.x are related to
>> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
>>
>> I don't see a good historical record to know if these are just flakes, but
>> if we only want to go on green builds perhaps we can either disable the
>> flakey tests or fix them up? If someone feels strongly we should fix
>> particular tests up please link a jira and I can take a whack at some of
>> them.
>>
>> -Joey
>>
>> On Tue, Jul 17, 2018 at 9:35 AM Michael Shuler 
>> wrote:
>>
>> > On 07/16/2018 11:27 PM, Jason Brown wrote:
>> > > Hey all,
>> > >
>> > > The recent builds were -1'd, but it appears the issues have been
>> resolved
>> > > (2.2.13 with CASSANDRA-14423, and 3.0.17 / 3.11.3 reverting
>> > > CASSANDRA-14252). Can we go ahead and reroll now?
>> >
>> > Could someone run through the tests on 2.2, 3.0, 3.11 branches and link
>> > them?  Thanks!
>> >
>> > Michael
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> > For additional commands, e-mail: dev-h...@cassandra.apache.org
>> >
>> >
>>
>
>

Re: reroll the builds?

[Jason Brown]

TL;DR We are in a better place than we were for the 3.0.16 and 3.11.2
releases. The current fails are not fatal, although they warrant
investigation. My opinion is that due the critical data loss bugs that are
fixed by CASSANDRA-14513 and CASSANDRA-14515, we should cut the builds now.

I've run the HEAD of the 3.0 and 3.11 branches vs the 3.0.16 and 3.11.2
release shas, and there are far less failing dtests now. In comparison:

- 3.11
-- HEAD - 5-6 failing tests
-- 3.11.2 - 18-20 failures

- 3.0
-- HEAD - 14-16 failures
-- 3.0.16 - 22-25 failures

The raw dump of my work can be found here:
https://gist.github.com/jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0. I've
applied no effort to clean it up, but it's available (includes links to the
circleci runs). I haven't completed an exhautive analysis of the failures
to see how far they go back as things become tricky (or, at least, very
time intensive to research) with the pytest/python-3 update with
CASSANDRA-14134. Thus some of the failures might be in the dtests
themselves (I suspect a couple of the failures are), but most are proabaly
legit failures.

As this thread is about cutting the releases, I'll save any significiant
analysis for a followup thread. I will say that the current failures are a
subset of the previous release's failures, and those failures are not data
loss bugs.

Overall, I feel far more comfortable getting the data loss fixes out
without any further delay than waiting for a few minor fixes. I will triage
the dtest failures over the coming days. There are some open tickets, and
I'll try to corral those with any new ones.

Thanks,

-Jason


On Mon, Jul 23, 2018 at 10:26 AM, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> I can help out with the triage / rerunning dtests if needed.
> Dinesh
>
> On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown <
> jasedbr...@gmail.com> wrote:
>
>  I spoke with some people over here, and I'm going to spend a day doing a
> quick triage of the failing dtests. There are some fixes for data loss bugs
> that are critical to get out in these builds, so I'll ensure the current
> failures are within an acceptable level of flakey-ness in order to unblock
> those fixes.
>
> Will have an update shortly ...
>
> -Jason
>
> On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown  wrote:
>
> > Hi all,
> >
> > First, thanks Joey for running the tests. Your pass/fail counts are
> > basically what in line with what I've seen for the last several months.
> (I
> > don't have an aggregated list anywhere, just observations from recent
> runs).
> >
> > Second, it's beyond me why there's such inertia to actually cutting a
> > release. We're getting up to almost *six months* since the last release.
> > Are there any grand objections at this point?
> >
> > Thanks,
> >
> > -Jason
> >
> >
> > On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> > wrote:
> >
> >> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there are
> >> various failing dtests but all three have green unit tests.
> >>
> >> 3.11.3 tentative (31d5d87, test branch
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/tree/
> >> cassandra_3.11_temp_testing>,
> >> unit tests <https://circleci.com/gh/vinaykumarchella/cassandra/258>
> >> pass, 5
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/256> and 6
> >> <https://circleci.com/gh/vinaykumarchella/cassandra/256#
> >> tests/containers/8>
> >> dtest failures)
> >> 3.0.17 tentative (d52c7b8, test branch
> >> <https://circleci.com/gh/jolynch/workflows/cassandra/tree/3.0-testing>,
> >> unit
> >> tests <https://circleci.com/gh/jolynch/cassandra/110> pass, 14
> >> <https://circleci.com/gh/jolynch/cassandra/112> and 15
> >> <https://circleci.com/gh/jolynch/cassandra/111> dtest failures)
> >> 2.2.13 tentative (3482370, test branch
> >> <https://circleci.com/gh/sumanth-pasupuleti/workflows/cassan
> >> dra/tree/2.2-testing>,
> >> unit tests <https://circleci.com/gh/sumanth-pasupuleti/cassandra/20>
> >> pass, 9
> >> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/21> and 10
> >> <https://circleci.com/gh/sumanth-pasupuleti/cassandra/22#
> >> tests/containers/8>
> >> dtest failures)
> >>
> >> It looks like many (~6) of the failures in 3.0.x are related to
> >> snapshot_test.TestArchiveCommitlog. I'm not sure if this is abnormal.
> >>
> >> I don't see a good hi

Re: reroll the builds?

[Jason Brown]

I did run the dtests against the last release shas (3.0.16 and 3.11.2).
Notes are all the way at the bottom of the gist about those runs. Circleci
URLs: https://circleci.com/workflow-run/5a1df5a1-f0c1-4ab4-a7db-e5551e7a5d38
/ https://circleci.com/workflow-run/a4369ab0-ae11-497a-8e10-de3995d10f25.

Current HEAD of 3.0 & 3.11 branches have significantly lower failing
dtests, and the failing tests on HEAD are a subset of those from the last
release.


On Tue, Jul 24, 2018 at 9:03 AM, dinesh.jo...@yahoo.com.INVALID <
dinesh.jo...@yahoo.com.invalid> wrote:

> Hi Jason,
> I agree - we should release with the dataloss bug fix. I went over the
> gist - apart from the Python errors and test teardown failures, there seem
> to be a few failures that look legitimate. Any chance you can run the
> dtests on the previous release SHAs and compare the dtest failures? If
> they're the same / similar, we know at least we're at parity with the
> previous release :)
> Dinesh
>
>     On Tuesday, July 24, 2018, 8:18:50 AM PDT, Jason Brown <
> jasedbr...@gmail.com> wrote:
>
>  TL;DR We are in a better place than we were for the 3.0.16 and 3.11.2
> releases. The current fails are not fatal, although they warrant
> investigation. My opinion is that due the critical data loss bugs that are
> fixed by CASSANDRA-14513 and CASSANDRA-14515, we should cut the builds now.
>
> I've run the HEAD of the 3.0 and 3.11 branches vs the 3.0.16 and 3.11.2
> release shas, and there are far less failing dtests now. In comparison:
>
> - 3.11
> -- HEAD - 5-6 failing tests
> -- 3.11.2 - 18-20 failures
>
> - 3.0
> -- HEAD - 14-16 failures
> -- 3.0.16 - 22-25 failures
>
> The raw dump of my work can be found here:
> https://gist.github.com/jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0. I've
> applied no effort to clean it up, but it's available (includes links to the
> circleci runs). I haven't completed an exhautive analysis of the failures
> to see how far they go back as things become tricky (or, at least, very
> time intensive to research) with the pytest/python-3 update with
> CASSANDRA-14134. Thus some of the failures might be in the dtests
> themselves (I suspect a couple of the failures are), but most are proabaly
> legit failures.
>
> As this thread is about cutting the releases, I'll save any significiant
> analysis for a followup thread. I will say that the current failures are a
> subset of the previous release's failures, and those failures are not data
> loss bugs.
>
> Overall, I feel far more comfortable getting the data loss fixes out
> without any further delay than waiting for a few minor fixes. I will triage
> the dtest failures over the coming days. There are some open tickets, and
> I'll try to corral those with any new ones.
>
> Thanks,
>
> -Jason
>
>
> On Mon, Jul 23, 2018 at 10:26 AM, dinesh.jo...@yahoo.com.INVALID <
> dinesh.jo...@yahoo.com.invalid> wrote:
>
> > I can help out with the triage / rerunning dtests if needed.
> > Dinesh
> >
> >On Monday, July 23, 2018, 10:22:18 AM PDT, Jason Brown <
> > jasedbr...@gmail.com> wrote:
> >
> >  I spoke with some people over here, and I'm going to spend a day doing a
> > quick triage of the failing dtests. There are some fixes for data loss
> bugs
> > that are critical to get out in these builds, so I'll ensure the current
> > failures are within an acceptable level of flakey-ness in order to
> unblock
> > those fixes.
> >
> > Will have an update shortly ...
> >
> > -Jason
> >
> > On Mon, Jul 23, 2018 at 9:18 AM, Jason Brown 
> wrote:
> >
> > > Hi all,
> > >
> > > First, thanks Joey for running the tests. Your pass/fail counts are
> > > basically what in line with what I've seen for the last several months.
> > (I
> > > don't have an aggregated list anywhere, just observations from recent
> > runs).
> > >
> > > Second, it's beyond me why there's such inertia to actually cutting a
> > > release. We're getting up to almost *six months* since the last
> release.
> > > Are there any grand objections at this point?
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > >
> > > On Tue, Jul 17, 2018 at 4:01 PM, Joseph Lynch 
> > > wrote:
> > >
> > >> We ran the tests against 3.0, 2.2 and 3.11 using circleci and there
> are
> > >> various failing dtests but all three have green unit tests.
> > >>
> > >> 3.11.3 tentative (31d5d87, test branch
> > >> <https://circleci.

Re: [VOTE] Release Apache Cassandra 2.2.13

[Jason Brown]

Ran the tests again, just for sanity sake, and all seems within acceptable
levels.

+1

On Wed, Jul 25, 2018 at 1:25 PM, Nate McCall  wrote:

> +1
>
> On Wed, Jul 25, 2018, 7:49 PM Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 2.2.13.
> >
> > sha1: 3482370df5672c9337a16a8a52baba53b70a4fe8
> > Git:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/2.2.13-tentative
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1167/org/apache/cassandra/apache-cassandra/2.2.13/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1167/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> > [2]: NEWS.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/2.2.13-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 3.11.3 (Take 2)

[Jason Brown]

Ran the tests again, just for sanity sake, and all seems within acceptable
levels (in order to get the data loss bugs resolved).

+1



On Wed, Jul 25, 2018 at 1:24 PM, Nate McCall  wrote:

> +1
>
> On Wed, Jul 25, 2018, 7:46 PM Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.11.3.
> >
> > sha1: 31d5d870f9f5b56391db46ba6cdf9e0882d8a5c0
> > Git:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.11.3-tentative
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1164/org/apache/cassandra/apache-cassandra/3.11.3/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1164/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
> > [2]: NEWS.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.11.3-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [VOTE] Release Apache Cassandra 3.0.17 (Take 2)

[Jason Brown]

Ran the tests again, just for sanity sake, and all seems within acceptable
levels (in order to get the data loss bugs resolved).

+1



On Wed, Jul 25, 2018 at 1:24 PM, Nate McCall  wrote:

> +1
>
> On Wed, Jul 25, 2018, 7:47 PM Michael Shuler 
> wrote:
>
> > I propose the following artifacts for release as 3.0.17.
> >
> > sha1: d52c7b8c595cc0d06fc3607bf16e3f595f016bb6
> > Git:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.0.17-tentative
> > Artifacts:
> >
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1165/org/apache/cassandra/apache-cassandra/3.0.17/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> orgapachecassandra-1165/
> >
> > The Debian and RPM packages are available here:
> > http://people.apache.org/~mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: CHANGES.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
> > [2]: NEWS.txt:
> >
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> blob_plain;f=CHANGES.txt;hb=refs/tags/3.0.17-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

triage of recent dtest failures

[Jason Brown]

Hi all,

Now that the vote is on for the next releases, I've done an initial triage
of the failed dtest runs from 2.2, 3.0, and 3.11 (based on my circleci
runs). Below are the ten dtest failures I found most often (not exhaustive,
but representative), and I opened up corresponding jiras for them.

Let's see if we can get all the tickets assigned and analyzed. If in
researching, a test is indescribably flakey, that is valuble knowledge as
well. Being able to identify flakey tests and label them as such is step
forward, lacking a full fix. With these tickets, it might be that the test
needs to be corrected, and might not necessarily be a problem with
database. In some cases, it will be a problem in casandra, so please keep
that in mind.

Remember, fixing these tests helps *all* of us get to more stable and
reliable releases, and thus makes the database and project as a whole
stronger.

Also, feel free to reach out to me with questions on these.

Thanks!

-Jason


* test_describecluster_more_information_three_datacenters -
nodetool_test.TestNodetool
- versions: 3.11, 3.0, 2.2
https://issues.apache.org/jira/browse/CASSANDRA-14484#


* test_closing_connections - thrift_hsha_test.TestThriftHSHA
- versions: 3.11, 3.0, 2.2
https://issues.apache.org/jira/browse/CASSANDRA-14595


* test_mutation_v5 - write_failures_test.TestWriteFailures
- versions: 3.11 only
https://issues.apache.org/jira/browse/CASSANDRA-14596


* snapshot_test.TestArchiveCommitlog.*
- versions: apparently only 3.0
https://issues.apache.org/jira/browse/CASSANDRA-14597


* test_decommissioned_node_cant_rejoin - topology_test.TestTopology
- versions: seen on 3.0, but may be more
https://issues.apache.org/jira/browse/CASSANDRA-14598


* test_functional - global_row_key_cache_test.TestGlobalRowKeyCache
- versions: apparently only 3.0
https://issues.apache.org/jira/browse/CASSANDRA-14599


* test_system_auth_ks_is_alterable - auth_test.TestAuth
- versions: 3.0 / 3.11
https://issues.apache.org/jira/browse/CASSANDRA-14600


* test_failure_threshold_deletions - paging_test.TestPagingWithDeletions
-versions 3.11 only
https://issues.apache.org/jira/browse/CASSANDRA-14601


* test_sstableofflinerelevel - offline_tools_test.TestOfflineTools
- versions 3.0 only
https://issues.apache.org/jira/browse/CASSANDRA-14602


* test_alter_rf_and_run_read_repair & test_read_repair_chance -
read_repair_test.TestReadRepair
- versions 2.2, 3.0
https://issues.apache.org/jira/browse/CASSANDRA-14603

Re: G1GC is now default

[Jason Brown]

Hi Jon,

Thanks for pointing that out. Looks like I missed that during the
CASSANDRA-9608 review. Robert and I didn't discuss that online or offline,
so it's something that snuck in.

I don't have a personal preference for GC algs right now, but at a minimum
it would probably be appropriate to return to CMS. We can certainly have a
discussion thread about switching the default, but until we have that that,
I think it's more honest to revert to CMS. If there's no immediate push
back on reverting, I'll have a patch within a day to revert.

Thanks,

-Jason

On Wed, Aug 8, 2018 at 3:34 PM, Jonathan Haddad  wrote:

> I fired up trunk to check something, and noticed this:
>
> INFO  [Service Thread] 2018-08-08 15:01:36,723 GCInspector.java:290 - G1
> Young Generation GC in 339ms.  G1 Eden Space: 4634705920 -> 0; G1 Old Gen:
> 1190138352 -> 1435504616; G1 Survivor Space: 406847488 -> 301989888;
>
> which I thought was a bit weird, since I was using trunk without making any
> changes and didn't remember seeing a JIRA where we agreed to make that
> change.  I looked back and saw it made it in as a side effect of
> CASSANDRA-9608, but wasn't explicitly discussed in the ticket, and there's
> no note of it in CHANGES.
>
> I'm personally OK with this change as G1 is a safer bet for anyone who uses
> the defaults, but we should be explicit about the change.  Can anyone think
> of a reason why we'd need to revert this back to ParNew / CMS?
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
>

Re: G1GC is now default

[Jason Brown]

restored CMS as the default GC alg in CASSANDRA-14636.

Thanks,

-Jason

On Wed, Aug 8, 2018 at 4:33 PM, Jason Brown  wrote:

> Hi Jon,
>
> Thanks for pointing that out. Looks like I missed that during the
> CASSANDRA-9608 review. Robert and I didn't discuss that online or offline,
> so it's something that snuck in.
>
> I don't have a personal preference for GC algs right now, but at a minimum
> it would probably be appropriate to return to CMS. We can certainly have a
> discussion thread about switching the default, but until we have that that,
> I think it's more honest to revert to CMS. If there's no immediate push
> back on reverting, I'll have a patch within a day to revert.
>
> Thanks,
>
> -Jason
>
> On Wed, Aug 8, 2018 at 3:34 PM, Jonathan Haddad  wrote:
>
>> I fired up trunk to check something, and noticed this:
>>
>> INFO  [Service Thread] 2018-08-08 15:01:36,723 GCInspector.java:290 - G1
>> Young Generation GC in 339ms.  G1 Eden Space: 4634705920 -> 0; G1 Old Gen:
>> 1190138352 -> 1435504616; G1 Survivor Space: 406847488 -> 301989888;
>>
>> which I thought was a bit weird, since I was using trunk without making
>> any
>> changes and didn't remember seeing a JIRA where we agreed to make that
>> change.  I looked back and saw it made it in as a side effect of
>> CASSANDRA-9608, but wasn't explicitly discussed in the ticket, and there's
>> no note of it in CHANGES.
>>
>> I'm personally OK with this change as G1 is a safer bet for anyone who
>> uses
>> the defaults, but we should be explicit about the change.  Can anyone
>> think
>> of a reason why we'd need to revert this back to ParNew / CMS?
>>
>> --
>> Jon Haddad
>> http://www.rustyrazorblade.com
>> twitter: rustyrazorblade
>>
>
>

upgrade guava on trunk before 9/1?

[Jason Brown]

Hey all,

Does anyone feel strongly about upgrading guava on trunk before the 9/1
feature freeze for 4.0? We are currently at 23.3 (thanks to
CASSANDRA-13997), and the current is 26.0.

I took a quick look, and there's about 17 compilation errors. They fall
into two categories, both of which appear not too difficult to resolve (I
didn't look too closely, tbh).

If anyone wants to tackle this LHF I can rustle up some review time.

Thanks,

-Jason

Re: upgrade guava on trunk before 9/1?

[Jason Brown]

Hey Ariel,

Tbqh, not that much. I was mostly thinking from the "I have conflicts on
guava versions in my app because I pull in cassandra and XYZ libraries, and
the transitive dependencies on guava use different versions" POV. Further,
we'll be on this version of guava for 4.0 for at least two years from now.

As I asked, "does anybody feeling strongly?". Personally, I'm sorta +0 to
+0.5, but I was just throwing this out there in case someone does really
think it best we upgrade (and wants to make a contribution).

-Jason




On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg  wrote:

> Hi,
>
> What do we get from Guava in exchange for upgrading?
>
> Ariel
>
> On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > Hey all,
> >
> > Does anyone feel strongly about upgrading guava on trunk before the 9/1
> > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > CASSANDRA-13997), and the current is 26.0.
> >
> > I took a quick look, and there's about 17 compilation errors. They fall
> > into two categories, both of which appear not too difficult to resolve (I
> > didn't look too closely, tbh).
> >
> > If anyone wants to tackle this LHF I can rustle up some review time.
> >
> > Thanks,
> >
> > -Jason
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Side Car New Repo vs not

[Jason Brown]

+1 for separate repo. For pretty much all the same reasons Aleksey
elucidated.

On Tue, Aug 21, 2018 at 10:20 AM, Aleksey Yeshchenko 
wrote:

> Sure, allow me to elaborate - at least a little bit. But before I do, just
> let me note that this wasn’t a veto -1, just a shorthand for “I don’t like
> this option”.
>
> It would be nice to have sidecar and C* version and release cycles fully
> decoupled. I know it *can* be done when in-tree, but the way we vote on
> releases with tags off current branches would have to change somehow.
> Probably painfully. It would be nice to be able to easily enforce freezes,
> like the upcoming one, on the whole C* repo, while allowing feature
> development on the sidecar. It would be nice to not have sidecar commits in
> emails from commits@ mailing list. It would be nice to not have C* CI
> trigger necessarily on sidecar commits. Groups of people working on the two
> repos will mostly be different too, so what’s the point in sharing the repo?
>
> Having an extra repo with its own set of branches is cheap and easy - we
> already do that with dtests. I like cleanly separated things when coupling
> is avoidable. As such I would prefer the sidecar to live in a separate new
> repo, while still being part of the C* project.
>
> —
> AY
>
> On 21 August 2018 at 17:06:39, sankalp kohli (kohlisank...@gmail.com)
> wrote:
>
> Hi Aleksey,
> Can you please elaborate on the reasons for your -1? This
> way we can make progress towards any one approach.
> Thanks,
> Sankalp
>
> On Tue, Aug 21, 2018 at 8:39 AM Aleksey Yeshchenko 
> wrote:
>
> > FWIW I’m strongly -1 on in-tree approach, and would much prefer a
> separate
> > repo, dtest-style.
> >
> > —
> > AY
> >
> > On 21 August 2018 at 16:36:02, Jeremiah D Jordan (
> > jeremiah.jor...@gmail.com) wrote:
> >
> > I think the following is a very big plus of it being in tree:
> > >> * Faster iteration speed in general. For example when we need to add
> a
> > >> new
> > >> JMX endpoint that the sidecar needs, or change something from JMX to
> a
> > >> virtual table (e.g. for repair, or monitoring) we can do all changes
> > >> including tests as one commit within the main repository and don't
> > have
> > >> to
> > >> commit to main repo, sidecar repo,
> >
> > I also don’t see a reason why the sidecar being in tree means it would
> not
> > work in a mixed version cluster. The nodes themselves must work in a
> mixed
> > version cluster during a rolling upgrade, I would expect any management
> > side car to operate in the same manor, in tree or not.
> >
> > This tool will be pretty tightly coupled with the server, and as
> someone
> > with experience developing such tightly coupled tools, it is *much*
> easier
> > to make sure you don’t accidentally break them if they are in tree. How
> > many times has someone updated some JMX interface, updated nodetool,
> and
> > then moved on? Breaking all the external tools not in tree, without
> > realizing it. The above point about being able to modify interfaces and
> the
> > side car in the same commit is huge in terms of making sure someone
> doesn’t
> > inadvertently break the side car while fixing something else.
> >
> > -Jeremiah
> >
> >
> > > On Aug 21, 2018, at 10:28 AM, Jonathan Haddad 
> > wrote:
> > >
> > > Strongly agree with Blake. In my mind supporting multiple versions is
> > > mandatory. As I've stated before, we already do it with Reaper, I'd
> > > consider it a major misstep if we couldn't support multiple with the
> > > project - provided admin tool. It's the same reason dtests are
> separate
> > -
> > > they work with multiple versions.
> > >
> > > The number of repos does not affect distribution - if we want to ship
> > > Cassandra with the admin / repair tool (we should, imo), that can be
> > part
> > > of the build process.
> > >
> > >
> > >
> > >
> > > On Mon, Aug 20, 2018 at 9:21 PM Blake Eggleston 
>
> > > wrote:
> > >
> > >> If the sidecar is going to be on a different release cadence, or
> > support
> > >> interacting with mixed mode clusters, then it should definitely be
> in
> > a
> > >> separate repo. I don’t even know how branching and merging would
> work
> > in a
> > >> repo that supports 2 separate release targets and/or mixed mode
> > >> compatibility, but I’m pretty sure it would be a mess.
> > >>
> > >> As a cluster management tool, mixed mode is probably going to be a
> goal
> > at
> > >> some point. As a new project, it will benefit from not being tied to
> > the C*
> > >> release cycle (which would probably delay any sidecar release until
> > >> whenever 4.1 is cut).
> > >>
> > >>
> > >> On August 20, 2018 at 3:22:54 PM, Joseph Lynch (joe.e.ly...@gmail.com)
>
> >
> > >> wrote:
> > >>
> > >> I think that the pros of incubating the sidecar in tree as a tool
> > first
> > >> outweigh the alternatives at this point of time. Rough tradeoffs
> that
> > I
> > >> see:
> > >>
> > >> Unique pros of in tree sidecar:
> > >> * Faster iteration speed in general. For example when we n

Re: Supporting multiple JDKs

[Jason Brown]

Some of our java8 code will not compile under java11 (see CASSANDRA-9608);
the symbols have literally been removed (Unsafe.monitorEnter() /
Unsafe.monitorExit() ). Setting -source to "8" will not help. Thus, we need
two compilers for the foreseeable future.



On Thu, Aug 23, 2018 at 3:44 PM, Sumanth Pasupuleti <
sumanth.pasupuleti...@gmail.com> wrote:

> I am not against using a compile-time quick-feedback tool like Animal
> Sniffer either. It is great to have such a tool to know of any obvious bad
> changes right away during development. However, in addition to using such a
> tool, I believe, when we make a release, we should build against the actual
> JDKs we support (that way, we are not making a release just based on the
> result of an external tool), and we should be able to optionally run UTs
> and DTests against the JDK  (i.e. Java7 and Java8 for C* 2.2).
>
> > What do we mean "support multiple JDKs" ?
> > Are we talking Run-time or Compile-time?
> I am talking both - compile-time can for Build, run-time for UTs and
> DTests.
>
> > I think that dtests are always going to be the important defence here,
> and that we simplify it by running dtests on a standardised JDK compile.
> I agree, but its good to have an optional workflow in CircleCI to be able
> to run DTest against the non-standardized JDK as well, which we support
> (Java7 for example, for C*2.2, and Java11 for C* 4.0)
>
> Thanks,
> Sumanth
>
> On Thu, Aug 23, 2018 at 12:06 AM Mick Semb Wever  wrote:
>
> >
> > > > There's a cost-optimisation here in reducing what we have to support.
> > >
> > > I agree and animal sniffer is a great way to ferret out obvious issues.
> > > I am not against using animal sniffer. I'm concerned that there are
> > > various incompatibilities[1] between JDK versions and I am not 100%
> > > certain that animal sniffer will catch all of them.
> >
> >
> > Yeah, but is compiling (and unit tests) really any more effective against
> > behavioural incompatibilities (which also occur from one patch JDK
> version
> > to the next)?
> >
> > I think that dtests are always going to be the important defence here,
> and
> > that we simplify it by running dtests on a standardised JDK compile.
> >
> > regards,
> > Mick
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [Discuss] Accept GoCQL driver donation

[Jason Brown]

I find this idea interesting and worth a strong discussion.

Something to consider is an argument floating around in the admin tool/side
car discussion: if we take an existing project wholesale, we inherit all of
it's design decision and technical debt (every project has these). On the
other hand, we also get working code that is running in production. I
haven't looked at the gocql code (and probably won't until we're a bit
beyond the 4.0 code freeze), so I can't speak to anything beyond a cursory
reading of the docs.

As we consider bringing in drivers under the project umbrella, we need to
evaluate each driver implementation's merit on a case-by-case basis. I'm
not sure how we divvy up that work, or whom to entrust with that work, but
it's something to keep in mind.

Thanks,

-Jason


On Fri, Aug 31, 2018 at 8:38 AM, Chris Bannister 
wrote:

> I intend to stay on and continue to contribute.
>
> On Fri, 31 Aug 2018 at 4:37 pm, Jonathan Ellis  wrote:
>
> > On Fri, Aug 31, 2018 at 9:14 AM Nate McCall  wrote:
> >
> > > Hi folks,
> > > So I was recently talking with, Chris Bannister  the gocql [0]
> > > maintainer, and he expressed an interest in donating the driver to the
> > > ASF.
> > >
> >
> > Is he looking to continue to maintain it or is he looking to give it a
> good
> > home when he moves on?
> >
> > We could accept this along the same lines as how we took in the dtest
> > > donation - going through the incubator IP clearance process [1], but
> > > in this case it's much simpler as an individual (Chris) owns the
> > > copyright.
> > >
> >
> > Is that actually the case?  Github says 128 contributors, and I don't see
> > any mention of a CLA in
> > https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.
> >
> > --
> > Jonathan Ellis
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: NGCC 2018?

[Jason Brown]

+1 to Jon's sentiment. Further, perhaps we should use that time after
GA'ing 4.0 to poll our users what they need/want from the next major
release of the database.

On Wed, Sep 5, 2018 at 9:31 AM, Jonathan Haddad  wrote:

> I'm thinking a month or two after 4.0 would give us time to unwind after
> the release and start to give real thought to big changes coming in the
> next release.  Let's focus on one thing at a time.
>
> On Wed, Sep 5, 2018 at 12:29 PM sankalp kohli 
> wrote:
>
> > A good time for NGCC will be closer to 4.0 release where we can plan what
> > we can put it on 4.0-next. I am not sure doing it now is going to help
> when
> > we are months away from 4.0 release.
> >
> > On Fri, Aug 31, 2018 at 7:42 AM Jay Zhuang 
> wrote:
> >
> > > Are we going to have a dev event next month? Or anything this year? We
> > may
> > > also be able to provide space in bay area and help to organize it.
> > (Please
> > > let us know, so we could get final approval for that).
> > >
> > > On Fri, Jul 27, 2018 at 10:05 AM Jonathan Haddad 
> > > wrote:
> > >
> > > > My interpretation of Nate's statement was that since there would be a
> > > bunch
> > > > of us at Lynn's event, we might as well do NGCC at the same time.
> > > >
> > > > On Thu, Jul 26, 2018 at 9:03 PM Ben Bromhead 
> > > wrote:
> > > >
> > > > > It sounds like there may be an appetite for something, but the NGCC
> > in
> > > > its
> > > > > current format is likely to not be that useful?
> > > > >
> > > > > Is a bay area event focused on C* developers something that is
> > > > interesting
> > > > > for the broader dev community? In whatever format that may be?
> > > > >
> > > > > On Tue, Jul 24, 2018 at 5:02 PM Nate McCall 
> > > wrote:
> > > > >
> > > > > > This was discussed amongst the PMC recently. We did not come to a
> > > > > > conclusion and there were not terribly strong feelings either
> way.
> > > > > >
> > > > > > I don't feel like we need to hustle to get "NGCC" in place,
> > > > > > particularly given our decided focus on 4.0. However, that should
> > not
> > > > > > stop us from doing an additional 'c* developer' event in sept. to
> > > > > > coincide with distributed data summit.
> > > > > >
> > > > > > On Wed, Jul 25, 2018 at 5:03 AM, Patrick McFadin <
> > pmcfa...@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > Ben,
> > > > > > >
> > > > > > > Lynn Bender had offered a space the day before Distributed Data
> > > > Summit
> > > > > in
> > > > > > > September (http://distributeddatasummit.com/) since we are
> both
> > > > > platinum
> > > > > > > sponsors. I thought he and Nate had talked about that being a
> > good
> > > > > place
> > > > > > > for NGCC since many of us will be in town already.
> > > > > > >
> > > > > > > Nate, now that I've spoken for you, you can clarify, :D
> > > > > > >
> > > > > > > Patrick
> > > > > > >
> > > > > >
> > > > > >
> > -
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > > > >
> > > > > > --
> > > > > Ben Bromhead
> > > > > CTO | Instaclustr 
> > > > > +1 650 284 9692
> > > > > Reliability at Scale
> > > > > Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
> > > > >
> > > >
> > > >
> > > > --
> > > > Jon Haddad
> > > > http://www.rustyrazorblade.com
> > > > twitter: rustyrazorblade
> > > >
> > >
> >
>
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
>

Re: UDF

[Jason Brown]

Hi Robert,

Thanks for taking on this work. Is this message a heads up that a patch is
coming/complete, or to spawn a discussion about including this in 4.0?

Thanks,

-Jason

On Tue, Sep 11, 2018 at 2:32 AM, Robert Stupp  wrote:

> In an effort to clean up our hygiene and limit the dependencies used by
> UDFs/UDAs, I think we should refactor the UDF code parts and remove the
> dependency to the Java Driver in that area without breaking existing
> UDFs/UDAs.
>
> A working prototype is in this branch: https://github.com/snazy/
> cassandra/tree/feature/remove-udf-driver-dep-trunk <
> https://github.com/snazy/cassandra/tree/feature/remove-
> udf-driver-dep-trunk> . The changes are rather trivial and provide 100%
> backwards compatibility for existing UDFs.
>
> The prototype copies the necessary parts from the Java Driver into the C*
> source tree to org.apache.cassandra.cql3.functions.types and adopts its
> usages - i.e. UDF/UDA code plus CQLSSTableWriter + StressCQLSSTableWriter.
> The latter two classes have a reference to UDF’s UDHelper and had to be
> changed as well.
>
> Some functionality, like type parsing & handling, is duplicated in the
> code base with this prototype - once in the “current” source tree and once
> for UDFs. However, unifying the code paths is not trivial, since the UDF
> sandbox prohibits the use of internal classes (direct and likely indirect
> dependencies).
>
> Robert
>
> —
> Robert Stupp
> @snazy
>
>

Re: Moving tickets out of 4.0 post freeze

[Jason Brown]

Before we blanket update all of these, let's make sure they are not
relevant for 4.0 - because some of them actually are. Some are docs, some
are parent tickets, some are testing.

Naively, here are some that are still 4.0:
CASSANDRA-14746 
CASSANDRA-13254 
CASSANDRA-14606 

Thanks,

-Jason

On Tue, Sep 25, 2018 at 4:05 AM, Benedict Elliott Smith  wrote:

> Do we want to manage more versions than we do now?  Why don’t we simply
> align these things to majors, like we’ve typically tried to anyway?
>
> I think it’s anyway better to decide on a strategy and find a versioning
> scheme that matches it, rather than to look for a strategy that matches our
> current versioning scheme.
>
>
>
>
> > On 25 Sep 2018, at 03:07, Mick Semb Wever  wrote:
> >
> >
> >
> >
> >> I'm totally spitballing here on possible uses of meaningful minors.
> >
> > Continuing the splitballing…
> >
> > What about aligning native protocol or sstable format changes with the
> minor version?
> >
> >
> >> Regardless, the OP's statement that new features and improvements
> should go to 4.0.x seems wrong
> >
> > Yeah, I instinctively thought features and improvements would be moved
> to either 4.x or 5.x (not to any subsequent patch version 4.0.x).
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Testing 4.0 upgrading

[Jason Brown]

Hi Tommy,

Thank you for taking the time to start kicking the tires on the upgrade to
4.0. It looks like you've found two bugs:

1) "Unknown column coordinator_port during deserialization" (reported on
3.x nodes)

- looks like the 4.0 node isn't filtering out a column from a system table
that 3.0 doesn't know about. Most likely due to CASSANDRA-7544. Can you
open a JIRA for this, and tag @aweisberg?

2) SSL connection problems

I unserstand the 4.0 -> 3.X connection problem, and documented it at [1] in
MessagingService. TL;DR we don't know the version of a peer when restarting
and need to wait for that peer to connect to the local node and pass it's
correct messaging version (if the local node cannot interpret the handhsake
from the peer). However, why for the inbound connection to the 4.0 node it
is seeing SSLv2 is unclear. Can you open a separate JIRA, and we'll go from
there? In the meantime, maybe enable the JDK's SSL debugging [2] on the 3.x
node to see exactly what it is trying to do? Also, you can enable wire
tracing on the 4.0 node by setting this value to true [3] and recompiling.
We can followup further in the jira.

Thanks!

-Jason

[1]
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/MessagingService.java#L1668
[2]
https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/ReadDebug.html
[3]
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/async/NettyFactory.java#L84

On Tue, Oct 23, 2018 at 2:44 AM Tommy Stendahl 
wrote:

> Hi,
>
> I have been testing upgrade to 4.0, I started out with a cluster with
> 3.0.15 and server encryption enabled. Due to some issues in my
> environment I did upgrade one of the nodes to 3.11.3, I think this
> turned out to be a good thing since I could observer the behaviour of
> upgrading from both 3.0.15 and 3.11.3 at the same time.
>
> At first I didn't have much success at all, it look like found multiple
> issues mostly with server encryption so I decided to simplify thing and
> disabled server encryption.
>
> So with server encryption disabled the upgrade was working ok, what I
> did notice was exceptions in the 3.0.15 and 3.11.3 nodes once the first
> 4.0 node started.
>
> 3.0.15 exception:
> 2018-10-22T11:05:38.883+0200 ERROR
> [MessagingService-Incoming-/10.216.193.244] CassandraDaemon.java:223
> Exception in thread Thread[MessagingService-Incoming-/10.216.193.244
> ,5,main]
> java.lang.RuntimeException: Unknown column coordinator_port during
> deserialization
>  at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:433)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.db.filter.ColumnFilter$Serializer.deserialize(ColumnFilter.java:447)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:647)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:584)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at org.apache.cassandra.net.MessageIn.read(MessageIn.java:98)
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:201)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:178)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:92)
>
> ~[apache-cassandra-3.0.15.jar:3.0.15]
>
> 3.11.3 exception:
> 2018-10-22T11:12:05.060+0200 ERROR
> [MessagingService-Incoming-/10.216.193.244] CassandraDaemon.java:228
> Exception in thread Thread[MessagingService-Incoming-/10.216.193.244
> ,5,main]
> java.lang.RuntimeException: Unknown column coordinator_port during
> deserialization
>  at
> org.apache.cassandra.db.Columns$Serializer.deserialize(Columns.java:452)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.db.filter.ColumnFilter$Serializer.deserialize(ColumnFilter.java:482)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:760)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.db.ReadCommand$Serializer.deserialize(ReadCommand.java:697)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.io.ForwardingVersionedSerializer.deserialize(ForwardingVersionedSerializer.java:50)
>
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at org.apache.cassandra.net.MessageIn.read(MessageIn.java:123)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
>  at
> org.apache.cassandra.net.IncomingTcpConnection.recei

Re: Revisit the proposal to use github PR

[Jason Brown]

I'm with Sylvain and Aleksey. Our multi-branch strategy does not work well
(if at all) with Github PR-style workflows (and, yes, I and most c*
maintainers have played this game). The cassandra-dtests repo is different
as we don't really branch there, so might be fine for Github PRs - except
for the merge commit noise, and that we prefer squashed patches for commit.

We could perhaps be a bit more explicit about "how to contribute" in the
docs [1], wrt branches as well as Github PRs (that we don't do them), but
it's mostly not unreasonable.

Thus, I'm in favor of the current process.

[1] http://cassandra.apache.org/doc/latest/development/patches.html


On Thu, Dec 13, 2018 at 9:20 AM Aleksey Yeschenko 
wrote:

> There are some nice benefits to GH PRs, one of them is that we could
> eventually set up CircleCI hooks that would explicitly prevent commits that
> don’t pass the tests.
>
> But handling multiple branches would indeed be annoying. Would have to
> either submit 1 PR per branch - which is both tedious and non-atomic - or
> do a mixed approach, with a PR for the oldest branch, then a manual merge
> upwards. The latter would be kinda meh, especially when commits for
> different branches diverge.
>
> For me personally, the current setup works quite well, and I mostly share
> Sylvain’s opinion above, for the same reasons listed.
>
> —
> AY
>
> > On 13 Dec 2018, at 08:15, Sylvain Lebresne  wrote:
> >
> > Fwiw, I personally find it very useful to have all discussion, review
> > comments included, in the same place (namely JIRA, since for better or
> > worse, that's what we use for tracking tickets). Typically, that means
> > everything gets consistently pushed to the  commits@ mailing list,
> which I
> > find extremely convenient to keep track of things. I also have a theory
> > that the inline-comments type of review github PR give you is very
> > convenient for nitpicks, shallow or spur-of-the-moment comments, but
> > doesn't help that much for deeper reviews, and that it thus to favor the
> > former kind of review.
> >
> > Additionally, and to Benedict's point, I happen to have first hand
> > experience with a PR-based process for a multi-branch workflow very
> similar
> > to the one of this project, and suffice to say that I hate it with a
> > passion.
> >
> > Anyway, very much personal opinion here.
> > --
> > Sylvain
> >
> >
> > On Thu, Dec 13, 2018 at 2:13 AM dinesh.jo...@yahoo.com.INVALID
> >  wrote:
> >
> >> I've been already using github PRs for some time now. Once you specify
> the
> >> ticket number, the comments and discussion are persisted in Apache Jira
> as
> >> work log so it can be audited if desired. However, committers usually
> >> squash and commit the changes once the PR is approved. We don't use the
> >> merge feature in github. I don't believe github we can merge the commit
> >> into multiple branches through the UI. We would need to merge it into
> one
> >> branch and then manually merge that commit into other branches. The big
> >> upside of using github PRs is that it makes collaborating a lot easier.
> >> Downside is that it makes it very difficult to follow along the
> progress in
> >> Apache Jira. The messages that github posts back include huge diffs and
> are
> >> aweful.
> >> Dinesh
> >>
> >>On Thursday, December 13, 2018, 1:10:12 AM GMT+5:30, Benedict Elliott
> >> Smith  wrote:
> >>
> >> Perhaps somebody could summarise the tradeoffs?  I’m a little concerned
> >> about how it would work for our multi-branch workflow.  Would we open
> >> multiple PRs?
> >>
> >> Could we easily link with external CircleCI?
> >>
> >> It occurs to me, in JIRA proposal mode, that an extra required field
> for a
> >> permalink to GitHub for the patch would save a lot of time I spend
> hunting
> >> for a branch in the comments.
> >>
> >>
> >>
> >>
> >>> On 12 Dec 2018, at 19:20, jay.zhu...@yahoo.com.INVALID wrote:
> >>>
> >>> It was discussed 1 year's ago:
> >> https://www.mail-archive.com/dev@cassandra.apache.org/msg11810.html
> >>> As all Apache projects are moving to gitbox:
> >> https://reference.apache.org/committer/github, should we revisit that
> and
> >> change our review/commit process to use github PR?A good example is
> >> Spark:"Changes to Spark source code are proposed, reviewed and committed
> >> via Github pull requests" (https://spark.apache.org/contributing.html).
> >>> /jay
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Revisit the proposal to use github PR

[Jason Brown]

To clarify my position: Github PRs are great for *reviewing* code, and the
commentary is much easier to follow imo. But for *merging* code, esp into
our multi-branch strategy, PRs don't fit well, unless there's some
technique I and perhaps others are unaware of.

On Thu, Dec 13, 2018 at 9:47 AM Ariel Weisberg  wrote:

> Hi,
>
> I'm not clear on what github makes worse. It preserves more history then
> the JIRA approach. When people invitably force push their branches you
> can't tell from the link to a branch on JIRA. Github preserves the comments
> and force push history so you know what version of the code each comment
> applied to. Github also tracks when requests for changes are acknowledged
> and resolved.  I have had to make the same change request many times and
> keep track independently whether it was resolved. This has also resulting
> in mistakes getting merged when I missed a comment that was ignored.
>
> Now that github can CC JIRA that also CCs to commits@. It's better then
> JIRA comments because each comment includes a small diff of the code the
> comment applies to. To do that in JIRA I have to manually link to the code
> the PR and most people don't do that for every comment so some of them are
> inscrutable after the fact. Also manually created links sometimes refer to
> references that disappear or get force pushed. It's a bit tricky to get
> right.
>
> To me arguing against leveraging a better code review workflow (whether
> github or some other tool) is like arguing against using source control
> tools. Sure the filesystem and home grown scripts can be used to work
> around lack of source control, but why would you?
>
> I see two complaints so far. One is that github PRs encourage nitpicking.
> I don't see tool based solution to that off the cuff. Another is that
> github doesn't by default CC JIRA. Maybe we can just refuse to accept
> improperly formatted PRs and look into auto rejecting ones that don't refer
> to a ticket?
>
> Ariel
>
> On Thu, Dec 13, 2018, at 12:20 PM, Aleksey Yeschenko wrote:
> > There are some nice benefits to GH PRs, one of them is that we could
> > eventually set up CircleCI hooks that would explicitly prevent commits
> > that don’t pass the tests.
> >
> > But handling multiple branches would indeed be annoying. Would have to
> > either submit 1 PR per branch - which is both tedious and non-atomic -
> > or do a mixed approach, with a PR for the oldest branch, then a manual
> > merge upwards. The latter would be kinda meh, especially when commits
> > for different branches diverge.
> >
> > For me personally, the current setup works quite well, and I mostly
> > share Sylvain’s opinion above, for the same reasons listed.
> >
> > —
> > AY
> >
> > > On 13 Dec 2018, at 08:15, Sylvain Lebresne  wrote:
> > >
> > > Fwiw, I personally find it very useful to have all discussion, review
> > > comments included, in the same place (namely JIRA, since for better or
> > > worse, that's what we use for tracking tickets). Typically, that means
> > > everything gets consistently pushed to the  commits@ mailing list,
> which I
> > > find extremely convenient to keep track of things. I also have a theory
> > > that the inline-comments type of review github PR give you is very
> > > convenient for nitpicks, shallow or spur-of-the-moment comments, but
> > > doesn't help that much for deeper reviews, and that it thus to favor
> the
> > > former kind of review.
> > >
> > > Additionally, and to Benedict's point, I happen to have first hand
> > > experience with a PR-based process for a multi-branch workflow very
> similar
> > > to the one of this project, and suffice to say that I hate it with a
> > > passion.
> > >
> > > Anyway, very much personal opinion here.
> > > --
> > > Sylvain
> > >
> > >
> > > On Thu, Dec 13, 2018 at 2:13 AM dinesh.jo...@yahoo.com.INVALID
> > >  wrote:
> > >
> > >> I've been already using github PRs for some time now. Once you
> specify the
> > >> ticket number, the comments and discussion are persisted in Apache
> Jira as
> > >> work log so it can be audited if desired. However, committers usually
> > >> squash and commit the changes once the PR is approved. We don't use
> the
> > >> merge feature in github. I don't believe github we can merge the
> commit
> > >> into multiple branches through the UI. We would need to merge it into
> one
> > >> branch and then manually merge that commit into other branches. The
> big
> > >> upside of using github PRs is that it makes collaborating a lot
> easier.
> > >> Downside is that it makes it very difficult to follow along the
> progress in
> > >> Apache Jira. The messages that github posts back include huge diffs
> and are
> > >> aweful.
> > >> Dinesh
> > >>
> > >>On Thursday, December 13, 2018, 1:10:12 AM GMT+5:30, Benedict
> Elliott
> > >> Smith  wrote:
> > >>
> > >> Perhaps somebody could summarise the tradeoffs?  I’m a little
> concerned
> > >> about how it would work for our multi-branch workflow.  Would we op

Re: [VOTE] Change Jira Workflow

[Jason Brown]

+1.

On Mon, Dec 17, 2018 at 7:36 AM Michael Shuler 
wrote:

> +1
>
> --
> Michael
>
> On 12/17/18 9:19 AM, Benedict Elliott Smith wrote:
> > I propose these changes <
> https://cwiki.apache.org/confluence/display/CASSANDRA/JIRA+Workflow+Proposals>*
> to the Jira Workflow for the project.  The vote will be open for 72 hours**.
> >
> > I am, of course, +1.
> >
> > * With the addendum of the mailing list discussion <
> https://lists.apache.org/thread.html/e4668093169aa4ef52f2bea779333f04a0afde8640c9a79a8c86ee74@%3Cdev.cassandra.apache.org%3E>;
> in case of any conflict arising from a mistake on my part in the wiki, the
> consensus reached by polling the mailing list will take precedence.
> > ** I won’t be around to close the vote, as I will be on vacation.
> Everyone is welcome to ignore the result until I get back in a couple of
> weeks, or if anybody is eager feel free to close the vote and take some
> steps towards implementation.
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Proposing an Apache Cassandra Management process

[Jason Brown]

It looks like we have consensus to move forward with the sidecar
I will be shepherding the project, and to that end, I'll create a repo
against which we can start working.

Thanks,

-Jason

On Fri, Nov 30, 2018 at 7:54 AM sankalp kohli 
wrote:

> If no one has more feedback, we should create JIRAs for all the subtasks
> discussed in the proposal. I can see JIRA for Bulk command but not others.
>
> On Mon, Nov 19, 2018 at 2:12 AM dinesh.jo...@yahoo.com.INVALID
>  wrote:
>
> > Thanks, Mick & Stefan for your inputs. What should we do as next steps to
> > move this proposal forward?
> > Dinesh
> >
> > On Sunday, November 18, 2018, 11:57:54 AM PST, Stefan Podkowinski <
> > s...@apache.org> wrote:
> >
> >  My goal for a side car would be to enable more people to contribute to
> > the project, by making it more accessible for anyone who’s not familiar
> > with the Cassandra code base, or not familiar with Java development in
> > general. Although most of the functionality described in the proposal
> > sounds useful to have, I’d already be happy to have a solid REST API for
> > the existing nodetool and JMX functionality. If an official side car,
> > installed separately on each node, would provide that, I’m sure we’d see
> > lots of new tools created by the community (web UIs, cli tools, ..)
> > based on that. This would also be a good foundation for other existing
> > tool to converge upon, e.g. by calling the REST APIs for repair
> > scheduling and progress tracking instead of JMX, or by continually
> > integrating and sharing useful helper calls. This would also give
> > Cassandra devs more leeway to replace some of the existing tooling
> > related code in Cassandra, e.g. by migrating to virtual tables, while at
> > the same time keep providing a stable API through the side car.
> >
> > What I’d also like to point out here is that implementing such a project
> > as an *official* side car, also implies to me having the same standards
> > when it comes to release quality. I’d also really prefer having feature
> > sets matching between Cassandra and the side car, e.g. authentication
> > and SSL should also be supported in the side car from the beginning,
> > ideally without any additional configuration.
> >
> >
> > On 06.11.18 10:40, Dinesh Joshi wrote:
> > > Hi all,
> > >
> > > Joey, Vinay & I have fleshed out the Management process proposal as the
> > very first CIP document (with Jason’s inputs). It is available on the
> cwiki
> > -
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652224
> > >
> > > Please comment on it and provide us any input that you may have. We
> want
> > to ideally time-box the period to 2 weeks so we avoid waiting
> indefinitely.
> > >
> > > Thanks,
> > >
> > > Dinesh
> > >
> > >> On Oct 22, 2018, at 7:30 AM, "dinesh.jo...@yahoo.com.INVALID" <
> > dinesh.jo...@yahoo.com.INVALID> wrote:
> > >>
> > >> Thanks for starting this, Mick. I will flesh it out.
> > >> Dinesh
> > >>
> > >>On Sunday, October 21, 2018, 1:52:10 AM PDT, Mick Semb Wever <
> > m...@apache.org> wrote:
> > >>
> > >>
> > >>> But I'll try to put together a strawman proposal for the doc(s) over
> > the
> > >>> weekend.
> > >>
> > >> I've thrown something quickly together here:
> > >> -
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201
> > >> -
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CIP-1%3A+Proposing+an+Apache+Cassandra+Management+process
> > >>
> > >> The former is a blatant rip-off from the Kafka and Spark design
> > proposal pages that Dinesh previously mentioned. I'd hoped to do more of
> an
> > analysis of the existing C* habits and precedence on design proposals
> > (implicit in jira tickets), but in lei of that this is a strawman to
> start
> > the discussion.
> > >>
> > >> The latter still needs to be fleshed out. Dinesh, can you do this? I
> > can add a subpage/section that describes the alternative/consuming
> > third-party tools out there.
> > >>
> > >> regards,
> > >> Mick
> > >>
> > >> -
> > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Sidecar repo created

[Jason Brown]

Hello all,

In moving the sidecar project forward, I've created a gitbox [1] repo for
the cassandra-sidecar which is located at:

git: https://gitbox.apache.org/repos/asf/cassandra-sidecar.git
github: https://github.com/apache/cassandra-sidecar

There are additional behaviors we can set on the repo [2]:


   1. Any Pull Request that gets opened, closed, reopened or commented on
   can get recorded on the project's mailing list
   2. If a project has a JIRA instance, any PRs or comments on PRs that
   include a JIRA ticket ID will trigger an update on that specific ticket
   (unless opted out of)
   3. GitHub activity can be relayed to IRC channels on the Freenode
   network.

I think we probably want to enable #1 & #2 to maintain parity with what we
have already with the main cassandra repo.

Thanks,

-Jason

[1] https://gitbox.apache.org/
[2] https://reference.apache.org/pmc/github

Re: Git Repo Migration

[Jason Brown]

+1 for the migration.

Thanks for owning and leading this, Sam

On Fri, Jan 4, 2019 at 02:58 Sam Tunnicliffe  wrote:

> As per the announcement on 7th December 2018[1], ASF infra are planning to
> shutdown the service behind git-wip-us.apache.org and migrate all
> existing repos to gitbox.apache.org
>
> There are further details in the original mail, but apparently one of the
> benefits of the migration is that we'll have full write access via Github,
> including the ability finally to close PRs. This affects the cassandra,
> cassandra-dtest and cassandra-build repos (but not the new
> cassandra-sidecar repo).
>
> A pre-requisite of the migration is to demonstrate consensus within the
> community, so to satisfy that formality I'm starting this thread to gather
> any objections or specific requests regarding the timing of the move.
>
> I'll collate responses in a week or so and file the necessary INFRA Jira.
>
> Thanks,
> Sam
>
> [1]
> https://lists.apache.org/thread.html/667772efdabf49a0a23d585539c127f335477e033f1f9b6f5079aced@%3Cdev.cassandra.apache.org%3E
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [VOTE] Release Apache Cassandra 2.2.0-rc1

[Jason Brown]

+1
On Friday, June 5, 2015, Sylvain Lebresne  wrote:

> +1
>
> On Fri, Jun 5, 2015 at 5:41 PM, Brandon Williams  > wrote:
>
> > +1
> >
> > On Fri, Jun 5, 2015 at 10:27 AM, Jake Luciani  > wrote:
> >
> > > I propose the following artifacts for release as 2.2.0-rc1.
> > >
> > > sha1: b0ae285bdc7377a64ed92f01c67ff46b40ecaac0
> > > Git:
> > >
> > >
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.2.0-rc1-tentative
> > > Artifacts:
> > >
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1059/org/apache/cassandra/apache-cassandra/2.2.0-rc1/
> > > Staging repository:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1059/
> > >
> > > The artifacts as well as the debian package are also available here:
> > > http://people.apache.org/~jake
> > >
> > > The vote will be open for 72 hours (longer if needed).
> > >
> > > [1]: http://goo.gl/B6LLNm (CHANGES.txt)
> > > [2]: http://goo.gl/mr638q (NEWS.txt)
> > >
> >
>

Re: [VOTE] Release Apache Cassandra 2.0.16

[Jason Brown]

+1

On Fri, Jun 19, 2015 at 6:00 AM, Aleksey Yeschenko 
wrote:

> +1
>
> --
> AY
>
> On June 19, 2015 at 15:56:38, Jake Luciani (j...@apache.org) wrote:
>
> I propose the following artifacts for release as 2.0.16.
>
> sha1: 23e66a9d1c50e4331e8c1d212c2eeb940c5471fa
> Git:
>
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.0.16-tentative
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1060/org/apache/cassandra/apache-cassandra/2.0.16/
> Staging repository:
> https://repository.apache.org/content/repositories/orgapachecassandra-1060/
>
> The artifacts as well as the debian package are also available here:
> http://people.apache.org/~jake
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: http://goo.gl/stVnvu (CHANGES.txt)
> [2]: http://goo.gl/nkYblK (NEWS.txt)
>

Re: [VOTE] Release Apache Cassandra 2.2.0

[Jason Brown]

+1

On Fri, Jul 17, 2015 at 4:02 PM, Aleksey Yeschenko 
wrote:

> +1
>
> --
> AY
>
> On July 17, 2015 at 21:08:06, Jake Luciani (j...@apache.org) wrote:
>
> I propose the following artifacts for release as 2.2.0.
>
> sha1: 437bb9de77f54aa5a4a6a634ab3d2c753a17b3fc
> Git:
>
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.2.0-tentative
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1067/org/apache/cassandra/apache-cassandra/2.2.0/
> Staging repository:
> https://repository.apache.org/content/repositories/orgapachecassandra-1067/
>
> The artifacts as well as the debian package are also available here:
> http://people.apache.org/~jake
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: http://goo.gl/3FbKhG (CHANGES.txt)
> [2]: http://goo.gl/sMGs53 (NEWS.txt)
>

Re: [VOTE] Release Apache Cassandra 3.0.0-alpha1

[Jason Brown]

+1

On Fri, Jul 31, 2015 at 6:45 AM, Jonathan Ellis  wrote:

> +1
>
> On Fri, Jul 31, 2015 at 8:42 AM, Jake Luciani  wrote:
>
> > I propose the following artifacts for release as 3.0.0-alpha1.
> >
> > sha1: b090ed6938c0fad792e51757384bd5ac7f35a301
> > Git:
> >
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.0-alpha1-tentative
> > Artifacts:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1070/org/apache/cassandra/apache-cassandra/3.0.0-alpha1/
> > Staging repository:
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1070/
> >
> > The artifacts as well as the debian package are also available here:
> > http://people.apache.org/~jake
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: http://goo.gl/oowQCQ (CHANGES.txt)
> > [2]: http://goo.gl/s0RHg4 (NEWS.txt)
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>

Re: [VOTE] Release Apache Cassandra 3.0.0

[Jason Brown]

+1

On Saturday, November 7, 2015, Josh McKenzie  wrote:

> +1
>
> On Fri, Nov 6, 2015 at 4:34 PM, Jake Luciani  > wrote:
>
> > I propose the following artifacts for release as 3.0.0.
> >
> > sha1: 96f407bce56b98cd824d18e32ee012dbb99a0286
> > Git:
> >
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.0-tentative
> > Artifacts:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1089/org/apache/cassandra/apache-cassandra/3.0.0/
> > Staging repository:
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1089/
> >
> > The artifacts as well as the debian package are also available here:
> > http://people.apache.org/~jake
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: http://goo.gl/0vghCL (CHANGES.txt)
> > [2]: http://goo.gl/xAElQy (NEWS.txt)
> >
>

Re: Proposal: deprecate Thrift now and remove support in 4.0

[Jason Brown]

+1. It's finally time.

On Mon, Dec 28, 2015 at 6:26 AM, Jonathan Ellis  wrote:

> Thrift has been officially frozen for almost two years and unofficially for
> longer. [1]  Meanwhile, maintaining Thrift support through changes like
> 8099 has been a substantial investment.
>
> I propose deprecating Thrift now and removing support in 4.0, i.e. Nov 2016
> if tick-tock goes as planned.
>
> I note that only 7% of our survey respondents [2] are using Thrift-only,
> and and those users are often on old releases (1.1 and 1.2), i.e. unlikely
> to upgrade to 4.x anyway.
>
> Another 20% of users are using a mix of Thrift and CQL.  Some have been
> unable to completely migrate because CQL doesn’t quite provide every
> feature from Thrift.  The last such outstanding issue is mixing static and
> dynamic Thrift “columns” in a single table.  We have an issue open to
> address this [3].
>
> I think it is reasonable to either deprecate Thrift immediately in 3.2 or
> to wait until 10857 is committed in 3.4.
>
> [1]
>
> http://mail-archives.apache.org/mod_mbox/cassandra-dev/201403.mbox/%3ccaldd-zim6knmr7f_zcpvpqk0b2g919tttathiuofnvlztaw...@mail.gmail.com%3E
>
> [2]
>
> https://docs.google.com/spreadsheets/d/1FegCArZgj2DNAjNkcXi1n2Y1Kfvf6cdZedkMPYQdvC0/edit#gid=0
>
> [3] https://issues.apache.org/jira/browse/CASSANDRA-10857
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>

Re: [VOTE] Release Apache Cassandra 3.2

[Jason Brown]

+1

On Sunday, January 10, 2016, Sylvain Lebresne  wrote:

> +1
>
> On Sat, Jan 9, 2016 at 12:32 PM, Robert Stupp  > wrote:
>
> > +1
> >
> > --
> > Robert Stupp
> > @snazy
> >
> >
> >
> > --
> > Robert Stupp
> > @snazy
> > > Am 08.01.2016 um 22:12 schrieb Jake Luciani  >:
> > >
> > > I propose the following artifacts for release as 3.2.
> > >
> > > sha1: 3c6dfa4aa0b9ffb0a48a02b949bff2a8406764e6
> > > Git:
> > >
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.2-tentative
> > > Artifacts:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1096/org/apache/cassandra/apache-cassandra/3.2/
> > > Staging repository:
> > >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1096/
> > >
> > > The artifacts as well as the debian package are also available here:
> > > http://people.apache.org/~jake
> > >
> > > The vote will be open for 72 hours (longer if needed).
> > >
> > > [1]: http://goo.gl/T8geYN (CHANGES.txt)
> > > [2]: http://goo.gl/VDwYyf (NEWS.txt)
> >
>

Re: [VOTE] Release Apache Cassandra 3.2.1

[Jason Brown]

+1

On Friday, January 15, 2016, Brandon Williams  wrote:

> +1
>
> On Thu, Jan 14, 2016 at 9:03 PM, Jake Luciani  > wrote:
>
> > I propose the following artifacts for release as 3.2.1.
> >
> > sha1: 2ac95bd6c5699a605e6f4256cb17b016c99e6dda
> > Git:
> >
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.2.1-tentative
> > Artifacts:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1097/org/apache/cassandra/apache-cassandra/3.2.1/
> > Staging repository:
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1097/
> >
> > The artifacts as well as the debian package are also available here:
> > http://people.apache.org/~jake
> >
> > The vote will be open for 24 hours (longer if needed).
> >
> > [1]: http://goo.gl/WG3TEx (CHANGES.txt)
> > [2]: http://goo.gl/z7OsrE (NEWS.txt)
> > [3]: https://goo.gl/MLo2Kg (DataStax Test Report)
> >
>

Re: [VOTE] Release Apache Cassandra 2.1.13

[Jason Brown]

+1

On Wed, Feb 3, 2016 at 5:44 AM, Jake Luciani  wrote:

> I propose the following artifacts for release as 2.1.13.
>
> sha1: d5b6d1b634f69709d2b3caa17cba52696ed2033d
> Git:
>
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.1.13-tentative
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1098/org/apache/cassandra/apache-cassandra/2.1.13/
> Staging repository:
> https://repository.apache.org/content/repositories/orgapachecassandra-1098/
>
> The artifacts as well as the debian package are also available here:
> http://people.apache.org/~jake
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: http://goo.gl/ldbtfT (CHANGES.txt)
> [2]: http://goo.gl/yToJdI (NEWS.txt)
> [3]: https://goo.gl/eWXZLZ (DataStax Test Report)
>

Re: [VOTE] Release Apache Cassandra 2.2.5

[Jason Brown]

+1

On Wed, Feb 3, 2016 at 6:21 AM, Aleksey Yeschenko 
wrote:

> +1
>
> --
> AY
>
> On 3 February 2016 at 16:51:35, Jake Luciani (j...@apache.org) wrote:
>
> I propose the following artifacts for release as 2.2.5.
>
> sha1: dd76858c7652541c7b137323f7b9e154686d6fba
> Git:
>
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.2.5-tentative
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1099/org/apache/cassandra/apache-cassandra/2.2.5/
> Staging repository:
> https://repository.apache.org/content/repositories/orgapachecassandra-1099/
>
> The artifacts as well as the debian package are also available here:
> http://people.apache.org/~jake
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: http://goo.gl/GVE9VN (CHANGES.txt)
> [2]: http://goo.gl/kbrReO (NEWS.txt)
> [3]: https://goo.gl/QCHBW3 (DataStax Test Report)
>

Re: A short guid on how to contribute patches to Cassandra

[Jason Brown]

Woo-hoo! Thanks :)

On Tue, Feb 9, 2016 at 5:59 AM, Aleksey Yeschenko 
wrote:

> Hello everyone,
>
> I’ve compiled a short guide for contributors (who aren’t committers yet)
> about how to properly contribute Cassandra patches:
>
>
> https://docs.google.com/document/d/1d_AzYQo74de9utbbpyXxW2w-b0__sFhC7b24ibiJqjw/edit?usp=sharing
>
> Following the outlined recommendations make the lives of committers much
> easier, without adding much hassle to contributor process.
>
> Follow the steps and feel the love.
>
> --
> AY

Re: [VOTE] Release Apache Cassandra 3.0.5

[Jason Brown]

+1

On Thu, Apr 7, 2016 at 5:17 PM, Aleksey Yeschenko 
wrote:

> +1
>
> --
> AY
>
> On 7 April 2016 at 23:11:17, Jake Luciani (j...@apache.org) wrote:
>
> I propose the following artifacts for release as 3.0.5.
>
> sha1: c6e6fa94d28c0d23a8154e3743c05b355dba710a
> Git:
>
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/3.0.5-tentative
> Artifacts:
>
> https://repository.apache.org/content/repositories/orgapachecassandra-1104/org/apache/cassandra/apache-cassandra/3.0.5/
> Staging repository:
> https://repository.apache.org/content/repositories/orgapachecassandra-1104/
>
> The artifacts as well as the debian package are also available here:
> http://people.apache.org/~jake
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: http://goo.gl/f2b0Di (CHANGES.txt)
> [2]: http://goo.gl/bHIeYj (NEWS.txt)
> [3]: https://goo.gl/noLkHT (DataStax Test Report)
>

Re: Research on scalability bug finder for Cassandra

[Jason Brown]

+1 to Tupshin's proposal: 10k nodes (massive clusters) really is the next
frontier.

I don't expect the vnodes to add that much to the gossip dissemination as
the tokens per-node are sent out only a handful of times (when a node joins
the ring, mostly). Without having hard data to back myself up, I'd suspect
that the Failure Detector fails first, as with clusters that large, we
probably can't propagate the heartbeats (via gossip) fast enough/regularly
enough so that peers start marking each other down (due to the Phi Accrual
alogrithm).

Thanks,

-Jason

Re: Welcome new committers Carl Yeksigian and Stefania Alborghetti!

[Jason Brown]

Woo-hoo! Congrats Carl and Stephania

On Thursday, April 21, 2016, Jonathan Ellis  wrote:

> Belated congratulations to Carl (added as committer in January) and
> Stefania (added this week)!
>
> Thanks for your hard work and we look forward to making Cassandra better
> with you!
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>

Re: Cassandra Java Driver and DataStax

[Jason Brown]

The client-server protocol is well defined in the Cassandra repo, so any
one may implement a client library for any language. However, it is a far
from trivial task, so not many folks build their own. Thus, already-built
drivers tend to become the de facto standard, but we (the Apache Cassandra
committers/PMC) do not/have not blessed any vendor's driver(s) as official.

As to why there is not a canonical set of drivers in the Cassandra repo,
well, we've just never gotten into that game as an OSS community.

-Jason (not affiliated with DataStax)

On Friday, June 3, 2016, Johan Edstrom  wrote:

>
> On Jun 3, 2016, at 9:14 PM, Jeff Jirsa  > wrote:
>
>
> https://github.com/hector-client/hector
>
>
> So - that isn’t doing CQL, Right?
>
>
> https://github.com/Netflix/astyanax
>
>
> Upgrading to CQL?
>
>
> http://doanduyhai.github.io/Achilles/
>
>
> Which driver do you use?
>
> https://github.com/noorq/casser
>
>
> 2.1.5
>
>
>
> https://github.com/impetus-opensource/Kundera
>
>
> 
> ds-driver
> 
> false
> 
> 
> cassandra-core
> cassandra-ds-driver
> 
> 
> 
> thrift
> 
> false
> 
> 
> cassandra-core
> cassandra-thrift
> 
> 
>
>
>
>
>
>
> https://github.com/deanhiller/playorm
>
>
>
>
>
>
> - Jeff ( Not affiliated with datastax )
>
>
>
>
>
> On 6/3/16, 7:58 PM, "Johan Edstrom"  > wrote:
>
> How many Java drivers could you point out?
> Doesn’t it strike you slightly off that you’d not have a driver for a DB
> in the same project you found the DB?
>
>
>
> On Jun 3, 2016, at 8:51 PM, Dave Brosius  > wrote:
>
> There are many drivers for cassandra, supplied by various individuals and
> groups, one of those drivers was started by people at datastax which is
> available as an opensource project.
>
> The open source project is not open to any random person on the internet
> to commit to (just like any open source project), so i suppose in that
> regard there is some 'control'. But i doubt that is what you are fishing
> for.
>
> --dave
>
> (not affiliated with datastax)
>
> On 06/03/2016 10:29 PM, Chris Mattmann wrote:
>
> Hi All,
>
> I’m investigating something a few ASF members contacted
> me about and pointed out, so I’m hoping you can help
> guide me here as a community. I have heard that a company,
> DataStax, whose marketing material mentions it as the only
> Cassandra vendor, “controls” the Java Driver for Apache
> Cassandra.
>
> Of course, no company “controls” our projects or its code,
> so I told the folks that mentioned it to me that I’d investigate
> with my board hat on.
>
> I’d like to hear the community’s thoughts here on this. Does
> anyone in the community see this “controlling” behavior going
> on? Please speak up, as I’d like to get to the bottom of it,
> and I’ll be around on the lists, doing some homework and reading
> up on the archives to see what’s up.
>
> Thanks for any help you can provide in rooting this out.
>
> Cheers,
> Chris
>
>
>
>
>
>
>
>

Re: [VOTE] Release Apache Cassandra 2.1.15

[Jason Brown]

+1

On Friday, July 1, 2016, Sam Tunnicliffe  wrote:

> +1
>
> On Fri, Jul 1, 2016 at 4:44 PM, Jonathan Ellis  > wrote:
>
> > +1 on the release, but let's get those fixed for next time.
> >
> > On Thu, Jun 30, 2016 at 6:02 PM, Dave Brosius  >
> > wrote:
> >
> > > nits
> > >
> > > snappy-java and thrift-server's license files have the wrong version
> > > numbers.
> > >
> > >
> > >
> > >
> > > On 06/30/2016 03:30 PM, Jake Luciani wrote:
> > >
> > >> I propose the following artifacts for release as 2.1.15.
> > >>
> > >> sha1: cb14186f8d6c2d1105a51e409c59a4e424958171
> > >> Git:
> > >>
> > >>
> >
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/2.1.15-tentative
> > >> Artifacts:
> > >>
> > >>
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1118/org/apache/cassandra/apache-cassandra/2.1.15/
> > >> Staging repository:
> > >>
> > >>
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1118/
> > >>
> > >> The artifacts as well as the debian package are also available here:
> > >> http://people.apache.org/~jake
> > >>
> > >> The vote will be open for 72 hours (longer if needed).
> > >>
> > >> [1]: http://goo.gl/Dah6P4 (CHANGES.txt)
> > >> [2]: http://goo.gl/ZFeiGF (NEWS.txt)
> > >>
> > >>
> > >
> >
> >
> > --
> > Jonathan Ellis
> > Project Chair, Apache Cassandra
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: Blockers for 4.0

[Jason Brown]

CASSANDRA-8457 - nio MessagingService. Patch is up and awaiting review

On Wed, Jul 20, 2016 at 7:40 AM, Jonathan Ellis  wrote:

> The plan of record has been to ship 4.0 in November, 12 months after 3.0.
> But, there are a number of features that are going to cause backwards
> incompatibility and if they miss 4.0 will need to wait for 5.0.  Are any of
> these worth delaying 4.0 for?
>
> (Currently the plan is to have all of these ready for November, but let's
> get our backup plan figured out now, just in case.  That way we don't have
> to make the decision at the last minute when everything feels like an
> emergency.)
>
> Some candidates that might be worth delaying the release for:
>
>-  "Birch" trees for the primary key index
>.  Changes the
>format of data on disk so automatically in the "dot zero" category.
>- Decouple messaging protocol versioning
>.  This would
>allow us to change the intra-node protocol on a per-message basis, which
>gives us more flexibility with compatibility.  Currently any change
> drops
>us into the "no schema changes until everyone is upgraded" world which
>effectively rules out making any improvements across tick-tock releases.
>- Allow dropping COMPACT STORAGE flag
>.  This is what
>makes it possible to remove the deprecated Thrift support.
>- Schema rearchitecture
>.  Can we live
>without safe and programatic CREATE and ALTER for another year?
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>

Re: Blockers for 4.0

[Jason Brown]

forgot to mention that 8457 changes the internode messaging protocol, so
needs to fall on a major version boundary.

If 8457 does go forward, and CASSANDRA-8911 (mutation-based repair) does
*not* happen, we'll need something like CASSANDRA-12229 (to support
streaming under the non-blocking/netty model due to the changes introduced
in 8457, which are documented in ticket)

On Wed, Jul 20, 2016 at 8:16 AM, Aleksey Yeschenko 
wrote:

> I’d strike CASSANDRA-10383 off the list - there is no way it’s a blocker
> for anything.
>
> As for 9424, unless I die unexpectedly *and* nobody else picks up the
> work, it should be fine for Nov.
>
> Don’t see anything missing from the list.
>
> --
> AY
>
> On 20 July 2016 at 15:59:34, Jason Brown (jasedbr...@gmail.com) wrote:
>
> CASSANDRA-8457 - nio MessagingService. Patch is up and awaiting review
>
> On Wed, Jul 20, 2016 at 7:40 AM, Jonathan Ellis  wrote:
>
> > The plan of record has been to ship 4.0 in November, 12 months after 3.0.
> > But, there are a number of features that are going to cause backwards
> > incompatibility and if they miss 4.0 will need to wait for 5.0. Are any
> of
> > these worth delaying 4.0 for?
> >
> > (Currently the plan is to have all of these ready for November, but let's
> > get our backup plan figured out now, just in case. That way we don't have
> > to make the decision at the last minute when everything feels like an
> > emergency.)
> >
> > Some candidates that might be worth delaying the release for:
> >
> > - "Birch" trees for the primary key index
> > <https://issues.apache.org/jira/browse/CASSANDRA-9754>. Changes the
> > format of data on disk so automatically in the "dot zero" category.
> > - Decouple messaging protocol versioning
> > <https://issues.apache.org/jira/browse/CASSANDRA-12042>. This would
> > allow us to change the intra-node protocol on a per-message basis, which
> > gives us more flexibility with compatibility. Currently any change
> > drops
> > us into the "no schema changes until everyone is upgraded" world which
> > effectively rules out making any improvements across tick-tock releases.
> > - Allow dropping COMPACT STORAGE flag
> > <https://issues.apache.org/jira/browse/CASSANDRA-10857>. This is what
> > makes it possible to remove the deprecated Thrift support.
> > - Schema rearchitecture
> > <https://issues.apache.org/jira/browse/CASSANDRA-9424>. Can we live
> > without safe and programatic CREATE and ALTER for another year?
> >
> > --
> > Jonathan Ellis
> > Project Chair, Apache Cassandra
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: Blockers for 4.0

[Jason Brown]

Sylvain,

In the large, yes, that is the best "have enough mechanism in place that no
further ticket _have to_ wait for a major", but many of the tickets we are
talking about makes changes to things we've all agreed can *only* happen at
majors, as per the http://wiki.apache.org/cassandra/CompatibilityGuarantees
(changing the internode protocol, for instance).

if we're willing to trade on those Guarantees, then sure, majors are just
bike shedding the version numbers, but I suspect we don't want to do that ;)

That being said, I completely understand the frustration of "one more week,
X is almost ready and we don't want to wait a year to get it in". I offer
two points, though:
1) This is an open source project, with no real business requirements to
ship any version by any specific date. We want to ship often enough to be
relevant as a project/product, of course, but there's no financial or
business requirement to do so.
2) We could have better planning as to what we actually want to ship in a
"version", or at least have some agreed upon mid- to long-term roadmap.
This would help focus the calendar and the project. Currently, it's largely
willy-nilly as to what ships when, and while tick-tock has introduced
regularity wrt release dates, there's not much that shapes what goes in
those releases.

My two cents,

-Jason



On Thu, Jul 21, 2016 at 7:02 AM, Sylvain Lebresne 
wrote:

> My very own preference would be to actually focus on making 4.0 the release
> where have enough mechanism in place that no further ticket _have to_ wait
> for a major. That means at least making sure CASSANDRA-12042 makes it in,
> adding some proper versioning of schemas and CASSANDRA-8110.
>
> If we had all that in place, I think no other would _have to_ be ready for
> 4.0 as it could then just be delayed to 4.2 (or whenever it's ready).
>
> If we don't do that, I'm willing to take bets that every new major release
> every year will be delayed cause "one more week, X is almost ready and we
> don't want to wait a year to get it in".
>
> On Wed, Jul 20, 2016 at 4:40 PM, Jonathan Ellis  wrote:
>
> > The plan of record has been to ship 4.0 in November, 12 months after 3.0.
> > But, there are a number of features that are going to cause backwards
> > incompatibility and if they miss 4.0 will need to wait for 5.0.  Are any
> of
> > these worth delaying 4.0 for?
> >
> > (Currently the plan is to have all of these ready for November, but let's
> > get our backup plan figured out now, just in case.  That way we don't
> have
> > to make the decision at the last minute when everything feels like an
> > emergency.)
> >
> > Some candidates that might be worth delaying the release for:
> >
> >-  "Birch" trees for the primary key index
> >.  Changes the
> >format of data on disk so automatically in the "dot zero" category.
> >- Decouple messaging protocol versioning
> >.  This would
> >allow us to change the intra-node protocol on a per-message basis,
> which
> >gives us more flexibility with compatibility.  Currently any change
> > drops
> >us into the "no schema changes until everyone is upgraded" world which
> >effectively rules out making any improvements across tick-tock
> releases.
> >- Allow dropping COMPACT STORAGE flag
> >.  This is
> what
> >makes it possible to remove the deprecated Thrift support.
> >- Schema rearchitecture
> >.  Can we live
> >without safe and programatic CREATE and ALTER for another year?
> >
> > --
> > Jonathan Ellis
> > Project Chair, Apache Cassandra
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: Reminder: critical fixes only in 2.1

[Jason Brown]

D'oh, too late to respond :)

I was going to comment that leaving the non-critical commits in 2.1 is
probably OK at this point (the deed has already been done), as long as we
agree to become more rigorous in the future about critical bugs vs bugs vs
minor enhancements vs major features and in which versions they land.




On Thu, Jul 21, 2016 at 7:37 AM, Jonathan Ellis  wrote:

> Sounds like we have consensus on that.  I will handle the reverts and
> update Jira.
>
> On Wed, Jul 20, 2016 at 7:11 PM, Aleksey Yeschenko 
> wrote:
>
> > Keep #11349, revert the rest sounds reasonable.
> >
> > --
> > AY
> >
> > On 20 July 2016 at 22:27:05, sankalp kohli (kohlisank...@gmail.com)
> wrote:
> >
> > +1 on only allowing critical bug fixes.
> > I agree with Sylvain that CASSANDRA-11349 is a border line critical bug.
> I
> > would vote for CASSANDRA-11349 as being critical since over streaming is
> a
> > big issue for us as well. I am also fine taking it as an internal patch
> > since we already maintain an internal branch with bug fixes.
> >
> > On Wed, Jul 20, 2016 at 2:10 PM, Fabien Rousseau 
> > wrote:
> >
> > > Hi,
> > >
> > > I'm the reporter for CASSANDRA-11349.
> > > Just a bit of history:
> > >
> > > This was a big pain point for us because the over-streaming was really
> > > important.
> > > Each week, repair increased data size by a few percents.
> > > To give an example, one cluster with 12 nodes had a CF with around
> 250GB
> > of
> > > data per node and after compaction, reduced to 80GB.
> > > Before fully understanding the problem, we just doubled the cluster
> size
> > > just because of this issue (from 6 nodes to 12 nodes).
> > > We spotted the problem in November 2015, in one cluster, without having
> > > hints of what was happening at first (we just knew that there was a lot
> > of
> > > streaming during repairs, ie a few MB/s of data, and that there was
> > quite a
> > > high ratio of mismatch on read repairs)
> > > After putting in production more clusters (with completely different
> data
> > > models), we continued to observe this pattern (without finding any
> > > correlation).
> > > It took some time to identify the issue and fix it.
> > > Note that, once we reported the issue, we stopped repairing the
> affected
> > > CFs. (I know it's a bad practice, but this was a temporary solution on
> > > bare-metal servers which are quite stable).
> > >
> > > I'd say that I'm surprised that this was not reported by more people
> (our
> > > data model has nothing particular).
> > > (Maybe other people are affected but have not paid attention to this
> > > phenomenon)
> > >
> > > Please note that this patch does not affect how SSTables are
> serialized,
> > > but only how digest are computed thus reduces risks.
> > >
> > > So, to conclude, it was critical for us, and it may help other people
> in
> > > the wild.
> > > Nevertheless, we can live with this patch not being included in 2.1.X
> (by
> > > maintaining our own C* package until we migrate to 3.0.X)
> > >
> > > 2016-07-20 18:06 GMT+02:00 Sylvain Lebresne :
> > >
> > > > Definitively agrees that CASSANDRA-10433 and CASSANDRA-12030 aren't
> > > > critical. In fact, they are both marked as "improvements" and
> "minor".
> > > I'm
> > > > to blame for their commit, so mea culpa. But to my defense, I've long
> > > > advocated for being stricter on sticking to critical-only fixes on
> old
> > > > releases and have long felt that this sentiment wasn't really shared
> in
> > > > practice by other devs/committers (could be my fault getting the
> wrong
> > > > impression, but I've lost track of the number of time were other
> > > > devs/committers argue for "just this time because it's really simple"
> > > when
> > > > those questions cam up), so I've partly gave up on arguing. I'm happy
> > to
> > > > get more consistent on this and +1 reverting those.
> > > >
> > > > I think CASSANDRA-11349 is the one possibly more debatable. Fabien on
> > the
> > > > ticket seemed to suggest that on his clusters the bug was make repair
> > > > unmanageable ("a few days after filing the bug, we decided to
> > temporarily
> > > > stop repairing some tables [...] which were heavily impacted by those
> > > > bugs"). He mention in particular having seen "around a hundred"
> > > mismatches
> > > > with the patch against "a few hundred of thousands" without. I
> suppose
> > > one
> > > > could make a case that having repair over-stream by 3 order of
> > magnitude
> > > in
> > > > some case is critical-ish. Note that I wouldn't oppose reverting this
> > > too,
> > > > as it doesn't fully meet *my* definition of critical, but I'm willing
> > to
> > > > accept my definition is on the stronger side of things and leave it
> in.
> > > >
> > > > --
> > > > Sylvain
> > > >
> > > > On Wed, Jul 20, 2016 at 5:29 PM, Aleksey Yeschenko <
> alek...@apache.org
> > >
> > > > wrote:
> > > >
> > > > > +1 from me (and I don’t see any resistance to it either).
> > > > >
> > > > > --
> > > > > AY
> > > 

Re: State of Unit tests (1 out of 100 passes in trunk)

[Jason Brown]

>> Any reason not to draw a line in the sand for the next stabilization
release (3.9)?

+1

Josh, add me to the "test fixers" queue, as well. However, I think the
authors of patches that break the build should also be on the hook for
fixing problems, as well.




On Wed, Jul 20, 2016 at 2:05 PM, Jonathan Ellis  wrote:

> Also +1 from me.  Any reason not to draw a line in the sand for the next
> stabilization release (3.9)?
>
> On Wed, Jul 20, 2016 at 3:11 PM, sankalp kohli 
> wrote:
>
> > Hi,
> > I am +1 for not releasing new version till we fix the unit tests.
> Yes I
> > am interested in being added to the queue to make testing better. I have
> > fixed a bunch of flaky tests in our internal build and will open source
> > them soon.
> >
> > Can the PMC give some guidance on making releases block on unit tests?
> >
> > Also I see many unit tests commented out by 8099 like
> StreamTransferTest. I
> > dont know the history there so want to know if this is intentional?
> >
> > Thanks,
> > Sankalp
> >
> > On Wed, Jul 20, 2016 at 8:35 AM, Josh McKenzie 
> > wrote:
> >
> > > I've recently started a hard push for getting test tickets prioritized
> > and
> > > assigned to various contributors, and I'm pushing for us to have a
> > > consistent green test board for 3.9. While I'm not on the PMC so can't
> > > binding -1 a release vote, I'd non-binding -1 3.9 if we don't have
> stable
> > > tests.
> > >
> > > After that release my focus is going to shift to trunk (3.10) for
> > > stabilization; it's definitely something that's become a higher
> priority
> > > within the past month or so.
> > >
> > > If you or anyone else on the dev list here would like to be included in
> > the
> > > test-fixing assignment queue to help speed up the process (and get to
> > know
> > > parts of the code-base you don't yet know!), please let me know and
> I'll
> > > start kicking out test fix tickets your way once triaged.
> > >
> > > Thanks.
> > >
> > > On Tue, Jul 19, 2016 at 7:08 PM, sankalp kohli  >
> > > wrote:
> > >
> > > > Hi All,
> > > >   I can see that unit tests are very flaky in trunk as well
> as
> > > > previous versions of Cassandra. Out of the last 100 builds(#949 to
> > #1049)
> > > > for trunk, I only saw 1 successful build.
> > > >
> > > > Is there any plan to fix the failing tests that I am not aware of? I
> > see
> > > > lots of tickets recently for failing dtests, and am hoping the unit
> > tests
> > > > can receive similar attention.
> > > >
> > > > Thanks,
> > > > Sankalp
> > > >
> > >
> >
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>

Re: A proposal to move away from Jira-centric development

[Jason Brown]

Chris,

Can you give a few examples of other healthy Apache projects which you feel
would be good example? Note: I'm not trying to bait the conversation, but
am genuinely interested in what other successful projects do.

Thanks

Jason

On Monday, August 15, 2016, Chris Mattmann  wrote:

> s/dev list followers//
>
> That’s (one of) the disconnect(s). It’s not *you the emboldened, powerful
> PMC*
> and then everyone else.
>
>
> On 8/15/16, 11:25 AM, "Jeremy Hanna"  > wrote:
>
> Regarding high level linking, if I’m in irc or slack or hipchat or a
> mailing list thread, it’s easy to reference a Jira ID and chat programs can
> link to it and bots can bring up various details.  I don’t think a hash id
> for a mailing list is as simple or memorable.
>
> A feature of a mailing list thread is that it can go in different
> directions easily.  The burden is that it will be harder to follow in the
> future if you’re trying to sort out implementation details.  So for high
> level discussion, the mailing list is great.  When getting down to the
> actual work and discussion about that focused work, that’s where a tool
> like Jira comes in.  Then it is reference-able in the changes.txt and other
> things.
>
> I think the approach proposed by Jonathan is a nice way to keep dev
> list followers informed but keeping ticket details focused.
>
> > On Aug 15, 2016, at 1:12 PM, Chris Mattmann  > wrote:
> >
> > How is it harder to point someone to mail?
> >
> > Have you seen lists.apache.org?
> >
> > Specifically:
> > https://lists.apache.org/list.html?dev@cassandra.apache.org
> >
> >
> >
> > On 8/15/16, 10:08 AM, "Jeremiah D Jordan"  > wrote:
> >
> >I like keeping things in JIRA because then everything is in one
> place, and it is easy to refer someone to it in the future.
> >But I agree that JIRA tickets with a bunch of design discussion
> and POC’s and such in them can get pretty long and convoluted.
> >
> >I don’t really like the idea of moving all of that discussion to
> email which makes it has harder to point someone to it.  Maybe a better
> idea would be to have a “design/POC” JIRA and an “implementation” JIRA.
> That way we could still keep things in JIRA, but the final decision would
> be kept “clean”.
> >
> >Though it would be nice if people would send an email to the dev
> list when proposing “design” JIRA’s, as not everyone has time to follow
> every JIRA ever made to see that a new design JIRA was created that they
> might be interested in participating on.
> >
> >My 2c.
> >
> >-Jeremiah
> >
> >
> >> On Aug 15, 2016, at 9:22 AM, Jonathan Ellis  > wrote:
> >>
> >> A long time ago, I was a proponent of keeping most development
> discussions
> >> on Jira, where tickets can be self contained and the threadless
> nature
> >> helps keep discussions from getting sidetracked.
> >>
> >> But Cassandra was a lot smaller then, and as we've grown it has
> become
> >> necessary to separate out the signal (discussions of new features
> and major
> >> changes) from the noise of routine bug reports.
> >>
> >> I propose that we take advantage of the dev list to perform that
> >> separation.  Major new features and architectural improvements
> should be
> >> discussed first here, then when consensus on design is achieved,
> moved to
> >> Jira for implementation and review.
> >>
> >> I think this will also help with the problem when the initial idea
> proves
> >> to be unworkable and gets revised substantially later after much
> >> discussion.  It can be difficult to figure out what the conclusion
> was, as
> >> review comments start to pile up afterwards.  Having that
> discussion on the
> >> list, and summarizing on Jira, would mitigate this.
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder, http://www.datastax.com
> >> @spyced
> >
> >
> >
> >
>
>
>
>
>

Re: [ANN]: Cert management with self-signed CA for Cassandra (and presumably other Java stuff)

[Jason Brown]

+1. I recently discovered that a coworker had built, more or less, the same
thing just a month or two ago for internal/testing uses. And I know I've
seen/heard it elsewhere, so yeah, this would be great!

On Monday, August 22, 2016, Nate McCall  wrote:

> > > Any reason to not include this in the docs/operating or as a utility in
> repo
> > > to make it easier for end users to find all information in one place?
> Know
> > > this has come up on other projects and we always fall into the same
> > > search/reply trap as well
> >
> > No, if there were consensus that was worthwhile, I would have no
> objections.
> >
>
> It's worthwhile. I've had to do this a couple of times with different CM
> systems as work-for-hire and it's always a PITA. A general purpose utility
> would be awesome.
>

Re: Github pull requests

[Jason Brown]

It seems to me we might get more contributions if we can lower the barrier
to participation. (see Jeff Beck's statement above)

+1 to Aleksey's sentiment about the Docs contributions.

On Fri, Aug 26, 2016 at 9:48 AM, Mark Thomas  wrote:

> On 26/08/2016 17:11, Aleksey Yeschenko wrote:
> > Mark, I, for one, will be happy with the level of GitHub integration
> that Spark has, formal or otherwise.
>
> If Cassandra doesn't already have it, that should be a simple request to
> infra.
>
> > As it stands right now, none of the committers/PMC members have any
> control over Cassandra Github mirror.
> >
> > Which, among other things, means that we cannot even close the
> erroneously opened PRs ourselves,
> > they just accumulate unless the PR authors is kind enough to close them.
> That’s really frustrating.
>
> No PMC currently has the ability to directly close PRs on GitHub. This
> is one of the things on the infra TODO list that is being looked at. You
> can close them via a commit comment that the ASF GitHub tooling picks up.
>
> Mark
>
>
> >
> > --
> > AY
> >
> > On 26 August 2016 at 17:07:29, Mark Thomas (ma...@apache.org) wrote:
> >
> > On 26/08/2016 16:33, Jonathan Ellis wrote:
> >> Hi all,
> >>
> >> Historically we've insisted that people go through the process of
> creating
> >> a Jira issue and attaching a patch or linking a branch to demonstrate
> >> intent-to-contribute and to make sure we have a unified record of
> changes
> >> in Jira.
> >>
> >> But I understand that other Apache projects are now recognizing a github
> >> pull request as intent-to-contribute [1] and some are even making github
> >> the official repo, with an Apache mirror, rather than the other way
> >> around. (Maybe this is required to accept pull requests, I am not sure.)
> >>
> >> Should we revisit our policy here?
> >
> > At the moment, the ASF Git repo is always the master, with GitHub as a
> > mirror. That allows push requests to be made via GitHub.
> >
> > Infra is exploring options for giving PMCs greater control over GitHub
> > config (including allowing GitHub to be the master with a golden copy
> > held at the ASF) but that is a work in progress.
> >
> > As far as intent to contribute goes, there does appear to be a trend
> > that the newer a project is to the ASF, the more formal the project
> > makes process around recording intent to contribute. (The same can be
> > said for other processes as well like Jira config.)
> >
> > It is worth noting that all the ASF requires is that there is an intent
> > to contribute. Anything that can be reasonably read that way is fine.
> > Many PMCs happily accept patches sent to the dev list (although they may
> > ask them to be attached to issues more so they don't get forgotten than
> > anything else). Pull requests are certainly acceptable.
> >
> > My personal recommendation is don't put more formal process in place
> > than you actually need. Social controls are a lot more flexible than
> > technical ones and generally have a much lower overhead.
> >
> > Mark
> >
> >>
> >> [1] e.g. https://github.com/apache/spark/pulls?q=is%3Apr+is%3Aclosed
> >
> >
>
>

Re: #cassandra-dev IRC logging

[Jason Brown]

+1. How/where will this run? Is there any apache infra that we can make use
of?

On Fri, Aug 26, 2016 at 10:48 AM, Jake Luciani  wrote:

> +1 so long as it filters out the join/leave stuff :)
>
> On Fri, Aug 26, 2016 at 1:42 PM, Jeff Jirsa 
> wrote:
>
> > There exists a #cassandra-dev IRC channel that’s historically been used
> by
> > developers discussing the project – while it’s public, it’s not archived,
> > and it’s not a mailing list. The ASF encourages all discussion to be
> > archived, and ideally, archived on a public mailing list.
> >
> >
> >
> > Jake suggested, and I want to propose on the list, that we copy a log of
> > that channel (minus join/part activity) to dev@ either daily or weekly.
> > We’ll need to make sure we comply with Freenode’s IRC logging policy, but
> > the project / developers receives the best of both worlds – fast, real
> time
> > chat but also public archives/visibility for people who aren’t online at
> a
> > given moment. The volume may be a bit higher than most of us have come
> > expect from the list, but it brings the project closer to doing things in
> > The Apache Way, and we can give it an easily-filtered subject for folks
> who
> > don’t want that noise.
> >
> >
> >
> >
> >
>
>
>
> --
> http://twitter.com/tjake
>

Re: Github pull requests

[Jason Brown]

D'oh, forgot to explicitly state that I am +1 one on the github PR proposal
:)

On Fri, Aug 26, 2016 at 11:07 AM, Jason Brown  wrote:

> It seems to me we might get more contributions if we can lower the barrier
> to participation. (see Jeff Beck's statement above)
>
> +1 to Aleksey's sentiment about the Docs contributions.
>
> On Fri, Aug 26, 2016 at 9:48 AM, Mark Thomas  wrote:
>
>> On 26/08/2016 17:11, Aleksey Yeschenko wrote:
>> > Mark, I, for one, will be happy with the level of GitHub integration
>> that Spark has, formal or otherwise.
>>
>> If Cassandra doesn't already have it, that should be a simple request to
>> infra.
>>
>> > As it stands right now, none of the committers/PMC members have any
>> control over Cassandra Github mirror.
>> >
>> > Which, among other things, means that we cannot even close the
>> erroneously opened PRs ourselves,
>> > they just accumulate unless the PR authors is kind enough to close
>> them. That’s really frustrating.
>>
>> No PMC currently has the ability to directly close PRs on GitHub. This
>> is one of the things on the infra TODO list that is being looked at. You
>> can close them via a commit comment that the ASF GitHub tooling picks up.
>>
>> Mark
>>
>>
>> >
>> > --
>> > AY
>> >
>> > On 26 August 2016 at 17:07:29, Mark Thomas (ma...@apache.org) wrote:
>> >
>> > On 26/08/2016 16:33, Jonathan Ellis wrote:
>> >> Hi all,
>> >>
>> >> Historically we've insisted that people go through the process of
>> creating
>> >> a Jira issue and attaching a patch or linking a branch to demonstrate
>> >> intent-to-contribute and to make sure we have a unified record of
>> changes
>> >> in Jira.
>> >>
>> >> But I understand that other Apache projects are now recognizing a
>> github
>> >> pull request as intent-to-contribute [1] and some are even making
>> github
>> >> the official repo, with an Apache mirror, rather than the other way
>> >> around. (Maybe this is required to accept pull requests, I am not
>> sure.)
>> >>
>> >> Should we revisit our policy here?
>> >
>> > At the moment, the ASF Git repo is always the master, with GitHub as a
>> > mirror. That allows push requests to be made via GitHub.
>> >
>> > Infra is exploring options for giving PMCs greater control over GitHub
>> > config (including allowing GitHub to be the master with a golden copy
>> > held at the ASF) but that is a work in progress.
>> >
>> > As far as intent to contribute goes, there does appear to be a trend
>> > that the newer a project is to the ASF, the more formal the project
>> > makes process around recording intent to contribute. (The same can be
>> > said for other processes as well like Jira config.)
>> >
>> > It is worth noting that all the ASF requires is that there is an intent
>> > to contribute. Anything that can be reasonably read that way is fine.
>> > Many PMCs happily accept patches sent to the dev list (although they may
>> > ask them to be attached to issues more so they don't get forgotten than
>> > anything else). Pull requests are certainly acceptable.
>> >
>> > My personal recommendation is don't put more formal process in place
>> > than you actually need. Social controls are a lot more flexible than
>> > technical ones and generally have a much lower overhead.
>> >
>> > Mark
>> >
>> >>
>> >> [1] e.g. https://github.com/apache/spark/pulls?q=is%3Apr+is%3Aclosed
>> >
>> >
>>
>>
>

Re: #cassandra-dev IRC logging

[Jason Brown]

@Dave ASFBot looks like a winner. If others are on board with this, I can
work on getting it up and going.

On Fri, Aug 26, 2016 at 11:27 AM, Dave Lester  wrote:

> +1. Check out ASFBot for logging IRC, along with other integrations.[1]
>
> ASFBot can also be used for record keeping for IRC meetings (example [3])
> which can automatically be sent to the appropriate apache mailing list. All
> other logs are archived online. [4] It’d easy enough to link to those
> archived logs via the website, etc.
>
> [1] http://wilderness.apache.org/manual.html <
> http://wilderness.apache.org/manual.html>
> [3] http://wilderness.apache.org/#meetings <http://wilderness.apache.org/
> #meetings>
> [4] http://wilderness.apache.org/channels/#logs-#aurora <
> http://wilderness.apache.org/channels/#logs-%23aurora>
>
> > On Aug 26, 2016, at 11:17 AM, Jason Brown  wrote:
> >
> > +1. How/where will this run? Is there any apache infra that we can make
> use
> > of?
> >
> > On Fri, Aug 26, 2016 at 10:48 AM, Jake Luciani  wrote:
> >
> >> +1 so long as it filters out the join/leave stuff :)
> >>
> >> On Fri, Aug 26, 2016 at 1:42 PM, Jeff Jirsa  >
> >> wrote:
> >>
> >>> There exists a #cassandra-dev IRC channel that’s historically been used
> >> by
> >>> developers discussing the project – while it’s public, it’s not
> archived,
> >>> and it’s not a mailing list. The ASF encourages all discussion to be
> >>> archived, and ideally, archived on a public mailing list.
> >>>
> >>>
> >>>
> >>> Jake suggested, and I want to propose on the list, that we copy a log
> of
> >>> that channel (minus join/part activity) to dev@ either daily or
> weekly.
> >>> We’ll need to make sure we comply with Freenode’s IRC logging policy,
> but
> >>> the project / developers receives the best of both worlds – fast, real
> >> time
> >>> chat but also public archives/visibility for people who aren’t online
> at
> >> a
> >>> given moment. The volume may be a bit higher than most of us have come
> >>> expect from the list, but it brings the project closer to doing things
> in
> >>> The Apache Way, and we can give it an easily-filtered subject for folks
> >> who
> >>> don’t want that noise.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> http://twitter.com/tjake
> >>
>
>

Re: #cassandra-dev IRC logging

[Jason Brown]

Ed, did you mean this to post this to the other active thread today, the
one about github pull requests? (just want to make sure I'm understanding
correctly :) )

On Fri, Aug 26, 2016 at 12:28 PM, Edward Capriolo 
wrote:

> One thing to watch out for. The way apache-gossip is setup the PR's get
> sent to the dev list. However the address is not part of the list so the
> project owners get an email asking to approve/reject every PR and comment
> on the PR.
>
> This is ok because we have a small quite group but you probably do not want
> that with the number of SCM changes in the cassandra project.
>
> On Fri, Aug 26, 2016 at 3:05 PM, Jeff Jirsa 
> wrote:
>
> > +1 to both as well
> >
> > On 8/26/16, 11:59 AM, "Tyler Hobbs"  wrote:
> >
> > >+1 on doing this and using ASFBot in particular.
> > >
> > >On Fri, Aug 26, 2016 at 1:40 PM, Jason Brown 
> > wrote:
> > >
> > >> @Dave ASFBot looks like a winner. If others are on board with this, I
> > can
> > >> work on getting it up and going.
> > >>
> > >> On Fri, Aug 26, 2016 at 11:27 AM, Dave Lester 
> > >> wrote:
> > >>
> > >> > +1. Check out ASFBot for logging IRC, along with other
> > integrations.[1]
> > >> >
> >
> >
>

Re: Github pull requests

[Jason Brown]

@Jeremiah, makes sense to send to commits@

On Fri, Aug 26, 2016 at 12:55 PM, Jeremiah D Jordan <
jeremiah.jor...@gmail.com> wrote:

> +1 for PR’s but if we start using them I think we should get them sent to
> commits@ instead of the dev@ they are currently sent to.
>
> -Jeremiah
>
> > On Aug 26, 2016, at 1:38 PM, Andres de la Peña 
> wrote:
> >
> > +1 to GitHub PRs, I think it will make things easier.
> >
> > El viernes, 26 de agosto de 2016, Jason Brown 
> > escribió:
> >
> >> D'oh, forgot to explicitly state that I am +1 one on the github PR
> proposal
> >> :)
> >>
> >> On Fri, Aug 26, 2016 at 11:07 AM, Jason Brown  >> > wrote:
> >>
> >>> It seems to me we might get more contributions if we can lower the
> >> barrier
> >>> to participation. (see Jeff Beck's statement above)
> >>>
> >>> +1 to Aleksey's sentiment about the Docs contributions.
> >>>
> >>> On Fri, Aug 26, 2016 at 9:48 AM, Mark Thomas  >> > wrote:
> >>>
> >>>> On 26/08/2016 17:11, Aleksey Yeschenko wrote:
> >>>>> Mark, I, for one, will be happy with the level of GitHub integration
> >>>> that Spark has, formal or otherwise.
> >>>>
> >>>> If Cassandra doesn't already have it, that should be a simple request
> to
> >>>> infra.
> >>>>
> >>>>> As it stands right now, none of the committers/PMC members have any
> >>>> control over Cassandra Github mirror.
> >>>>>
> >>>>> Which, among other things, means that we cannot even close the
> >>>> erroneously opened PRs ourselves,
> >>>>> they just accumulate unless the PR authors is kind enough to close
> >>>> them. That’s really frustrating.
> >>>>
> >>>> No PMC currently has the ability to directly close PRs on GitHub. This
> >>>> is one of the things on the infra TODO list that is being looked at.
> You
> >>>> can close them via a commit comment that the ASF GitHub tooling picks
> >> up.
> >>>>
> >>>> Mark
> >>>>
> >>>>
> >>>>>
> >>>>> --
> >>>>> AY
> >>>>>
> >>>>> On 26 August 2016 at 17:07:29, Mark Thomas (ma...@apache.org
> >> ) wrote:
> >>>>>
> >>>>> On 26/08/2016 16:33, Jonathan Ellis wrote:
> >>>>>> Hi all,
> >>>>>>
> >>>>>> Historically we've insisted that people go through the process of
> >>>> creating
> >>>>>> a Jira issue and attaching a patch or linking a branch to
> demonstrate
> >>>>>> intent-to-contribute and to make sure we have a unified record of
> >>>> changes
> >>>>>> in Jira.
> >>>>>>
> >>>>>> But I understand that other Apache projects are now recognizing a
> >>>> github
> >>>>>> pull request as intent-to-contribute [1] and some are even making
> >>>> github
> >>>>>> the official repo, with an Apache mirror, rather than the other way
> >>>>>> around. (Maybe this is required to accept pull requests, I am not
> >>>> sure.)
> >>>>>>
> >>>>>> Should we revisit our policy here?
> >>>>>
> >>>>> At the moment, the ASF Git repo is always the master, with GitHub as
> a
> >>>>> mirror. That allows push requests to be made via GitHub.
> >>>>>
> >>>>> Infra is exploring options for giving PMCs greater control over
> GitHub
> >>>>> config (including allowing GitHub to be the master with a golden copy
> >>>>> held at the ASF) but that is a work in progress.
> >>>>>
> >>>>> As far as intent to contribute goes, there does appear to be a trend
> >>>>> that the newer a project is to the ASF, the more formal the project
> >>>>> makes process around recording intent to contribute. (The same can be
> >>>>> said for other processes as well like Jira config.)
> >>>>>
> >>>>> It is worth noting that all the ASF requires is that there is an
> >> intent
> >>>>> to contribute. Anything that can be reasonably read that way is fine.
> >>>>> Many PMCs happily accept patches sent to the dev list (although they
> >> may
> >>>>> ask them to be attached to issues more so they don't get forgotten
> >> than
> >>>>> anything else). Pull requests are certainly acceptable.
> >>>>>
> >>>>> My personal recommendation is don't put more formal process in place
> >>>>> than you actually need. Social controls are a lot more flexible than
> >>>>> technical ones and generally have a much lower overhead.
> >>>>>
> >>>>> Mark
> >>>>>
> >>>>>>
> >>>>>> [1] e.g. https://github.com/apache/spark/pulls?q=is%3Apr+is%
> 3Aclosed
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>
> >
> >
> > --
> > Andrés de la Peña
> >
> > Vía de las dos Castillas, 33, Ática 4, 3ª Planta
> > 28224 Pozuelo de Alarcón, Madrid
> > Tel: +34 91 828 6473 // www.stratio.com // *@stratiobd
> > <https://twitter.com/StratioBD>*
>
>

Gossip 2.0

[Jason Brown]

Hi all,

One of the problems I been thinking about for quite a while is how to
improve the efficacy of cassandra's gossip, especially when dealing with
large clusters (greater than 1,000 nodes). By reducing gossip overhead we
can implement new features that require data to be disseminated
cluster-wide and we can support those features to thousands of nodes. To
that end, I've done much research and thinking, and have opened up
CASSANDRA-12345 as a parent ticket for this work. I'd like to invite the
community to discuss the ideas, the approach, and hopefully dig into the
implementations that I've posted in the child tickets.

Thanks,

-Jason

Re: Support Multi-Tenant in Cassandra

[Jason Brown]

The current implementation will probably be yanked when thrift as a whole
is removed for 4.0. And I'm ok with that.

That being said, there has been an undercurrent of interest over time about
multitenancy, and I'm willing to entertain a renewed discussion. It might
be instructive to see if any other systems are currently offering
multitenancy and if there's something to be learned there. If not, we could
at least explore the topic more seriously and then document for posterity
the well-informed pros/cons of why we as a community choose to not do it,
postpone for later, or actually do it. Of course, it would be great for a
motivated individual to lead the effort if we really want to entertain it.

On Friday, September 9, 2016, Jeremy Hanna 
wrote:

> I agree that the request scheduler should probably be deprecated and
> removed unless someone wants to put in something that's usable from the non
> thrift request processor. We added it for prioritization and QoS but I
> don't know of anyone ever using it. Our project we thought of using it for
> got shelved.
>
> Unless it's just multiple clients with the same general use case, I think
> multi tenant is going to be quite difficult to tune and diagnose problems
> for. I would steer clear and have a cluster per logical app if at all
> possible.
>
> > On Sep 9, 2016, at 6:43 PM, Mick Semb Wever  > wrote:
> >
> > On 15 July 2016 at 16:38, jason zhao yang  >
> > wrote:
> >
> >>
> >> May I ask is there any plan of extending functionalities related to
> >> Multi-Tenant?
> >
> >
> >
> > I had needs for this in the past and my questioning always seemed to
> > eventuate to answers along the lines of this should be done more at the
> > resource level. There is a variety of ways a bad datamodel or client can
> > bring a cluster down, not just at request time.
> >
> > There was some thoughts IIRC around a resource scheduler somewhere
> post-3.0
> > but i don't think that ever eventuated (someone more knowledgable please
> > correct me).
> >
> > Otherwise you could look into using tiered storage so that you had at
> least
> > disk isolation per keyspace. Solves some things, but won't help with
> > overhead and memtable impact from number of keyspaces/tables and lack of
> > heap/throughput isolation/scheduling.
> >
> > The approach of doing this at the driver level, prefixing the partition
> > key, is as good as any approach for now.
> >
> > Could be an idea to remove/deprecate the request_scheduler from code and
> > yaml.
> >
> > ~mck
>

Re: Support Multi-Tenant in Cassandra

[Jason Brown]

Heh, nice find, Jeremy. Thanks for digging it up

On Friday, September 9, 2016, Jeremy Hanna 
wrote:

> For posterity, our wiki page from many moons ago was
> https://wiki.apache.org/cassandra/MultiTenant <https://wiki.apache.org/
> cassandra/MultiTenant>.  It was a different era of the project but there
> might be some useful bits in there for anyone interested in MT.
>
> > On Sep 9, 2016, at 9:28 PM, Jason Brown  > wrote:
> >
> > The current implementation will probably be yanked when thrift as a whole
> > is removed for 4.0. And I'm ok with that.
> >
> > That being said, there has been an undercurrent of interest over time
> about
> > multitenancy, and I'm willing to entertain a renewed discussion. It might
> > be instructive to see if any other systems are currently offering
> > multitenancy and if there's something to be learned there. If not, we
> could
> > at least explore the topic more seriously and then document for posterity
> > the well-informed pros/cons of why we as a community choose to not do it,
> > postpone for later, or actually do it. Of course, it would be great for a
> > motivated individual to lead the effort if we really want to entertain
> it.
> >
> > On Friday, September 9, 2016, Jeremy Hanna  >
> > wrote:
> >
> >> I agree that the request scheduler should probably be deprecated and
> >> removed unless someone wants to put in something that's usable from the
> non
> >> thrift request processor. We added it for prioritization and QoS but I
> >> don't know of anyone ever using it. Our project we thought of using it
> for
> >> got shelved.
> >>
> >> Unless it's just multiple clients with the same general use case, I
> think
> >> multi tenant is going to be quite difficult to tune and diagnose
> problems
> >> for. I would steer clear and have a cluster per logical app if at all
> >> possible.
> >>
> >>> On Sep 9, 2016, at 6:43 PM, Mick Semb Wever  
> >> > wrote:
> >>>
> >>> On 15 July 2016 at 16:38, jason zhao yang  
> >> >
> >>> wrote:
> >>>
> >>>>
> >>>> May I ask is there any plan of extending functionalities related to
> >>>> Multi-Tenant?
> >>>
> >>>
> >>>
> >>> I had needs for this in the past and my questioning always seemed to
> >>> eventuate to answers along the lines of this should be done more at the
> >>> resource level. There is a variety of ways a bad datamodel or client
> can
> >>> bring a cluster down, not just at request time.
> >>>
> >>> There was some thoughts IIRC around a resource scheduler somewhere
> >> post-3.0
> >>> but i don't think that ever eventuated (someone more knowledgable
> please
> >>> correct me).
> >>>
> >>> Otherwise you could look into using tiered storage so that you had at
> >> least
> >>> disk isolation per keyspace. Solves some things, but won't help with
> >>> overhead and memtable impact from number of keyspaces/tables and lack
> of
> >>> heap/throughput isolation/scheduling.
> >>>
> >>> The approach of doing this at the driver level, prefixing the partition
> >>> key, is as good as any approach for now.
> >>>
> >>> Could be an idea to remove/deprecate the request_scheduler from code
> and
> >>> yaml.
> >>>
> >>> ~mck
> >>
>
>

Re: [VOTE] Release Apache Cassandra 3.0.9

[Jason Brown]

+1

On Thu, Sep 15, 2016 at 3:20 PM, Nate McCall  wrote:

> +1
>
> On Fri, Sep 16, 2016 at 6:57 AM, Jake Luciani  wrote:
>
> > I propose the following artifacts for release as 3.0.9.
> >
> > sha1: d600f51ee1a3eb7b30ce3c409129567b70c22012
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/3.0.9-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1124/org/apache/cassandra/apache-cassandra/3.0.9/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1124/
> >
> > The artifacts as well as the debian package are also available here:
> > http://people.apache.org/~jake
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: https://goo.gl/JKkE05 (CHANGES.txt)
> > [2]: https://goo.gl/Hi8X71 (NEWS.txt)
> >
>
>
>
> --
> -
> Nate McCall
> Wellington, NZ
> @zznate
>
> CTO
> Apache Cassandra Consulting
> http://www.thelastpickle.com
>

Re: [VOTE] Release Apache Cassandra 2.2.8

[Jason Brown]

+1

On Friday, September 23, 2016, Brandon Williams  wrote:

> +1
>
> On Fri, Sep 23, 2016 at 6:04 PM, Michael Shuler  > wrote:
>
> > I propose the following artifacts for release as 2.2.8.
> >
> > sha1: e9fe96f404b6a936ac5dbceb8f3934fe0d098a97
> > Git:
> > http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> > shortlog;h=refs/tags/2.2.8-tentative
> > Artifacts:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1125/org/apache/cassandra/apache-cassandra/2.2.8/
> > Staging repository:
> > https://repository.apache.org/content/repositories/
> > orgapachecassandra-1125/
> >
> > The Debian packages are available here: http://people.apache.org/~
> mshuler
> >
> > The vote will be open for 72 hours (longer if needed).
> >
> > [1]: (CHANGES.txt) https://goo.gl/3Chc0Z
> > [2]: (NEWS.txt) https://goo.gl/0UgwXZ
> >
>

Re: [VOTE] Accept dtests Donation Into Project

[Jason Brown]

+1

On Fri, Sep 30, 2016 at 11:53 AM, Brandon Williams  wrote:

> +1
>
> On Fri, Sep 30, 2016 at 1:51 PM, Nate McCall  wrote:
>
> > I propose we begin the process of accepting the contribution of the
> > dtest codebase (https://github.com/riptano/cassandra-dtest) into the
> > project.
> >
> > Background discussion thread here:
> > https://lists.apache.org/thread.html/840fd900fb7f6568bfa008d122d437
> > 5b708c1f7f1b5929018118d5d5@%3Cdev.cassandra.apache.org%3E
> >
> > Note: It won't be immediate as there are some steps to follow [0] for
> > accepting outside code contributions.
> >
> > The vote will be open for 72 hours.
> >
> > -Nate
> >
> > [0] http://incubator.apache.org/ip-clearance/ip-clearance-template.html
> >
>

Re: [VOTE] Release Apache Cassandra 2.1.16

[Jason Brown]

+1

On Thu, Oct 6, 2016 at 3:45 PM, Romain Hardouin  wrote:

> Hi,
> I use the "current 2.1.16" (commit cdd535fcac4ba79bb371e8373c6504d9e3978853) 
> on
> production in 5 DCs (82 nodes) out of 7 and it works well!I've just had to
> add a MBean to track changes of the NTR queue length on top of cdd535f.
> This allow to make correlations with other metrics and see the impact of a
> change.
> I've filed a ticket with patches for 2.1 and the trunk
> https://issues.apache.org/jira/browse/CASSANDRA-12758
> Do you think this MBean could land in the final 2.1.16 since it goes
> hand-in-hand with CASSANDRA-11363?
>
> Thanks,
> Romain
>

Re: [VOTE] Release Apache Cassandra 2.1.16

[Jason Brown]

Ahh, sorry. I was +1'ing the proposed sha1 of 87034cd


On Friday, October 7, 2016, Michael Shuler  wrote:

> On 10/07/2016 08:52 AM, Jason Brown wrote:
> > +1
>
> Jason, is this a vote for the proposed sha1: 87034cd release for 2.1.16,
> or a comment to Romain? (gmail lacks some In-Reply-To/References
> intelligence sometimes, so just clarifying)
>
> > On Thu, Oct 6, 2016 at 3:45 PM, Romain Hardouin
>  >> wrote:
> >
> >> I've filed a ticket with patches for 2.1 and the trunk
> >> https://issues.apache.org/jira/browse/CASSANDRA-12758
> >> Do you think this MBean could land in the final 2.1.16 since it goes
> >> hand-in-hand with CASSANDRA-11363?
>
> I commented on CASSANDRA-12758 yesterday and adjusted fixver to 2.2.x+,
> but had not seen this list message until this morning.
>
> --
> Kind regards,
> Michael
>
>

Re: Rough roadmap for 4.0

[Jason Brown]

Hey Nate,

I'd like to add CASSANDRA-11559 (Enhance node representation) to the list,
including the comment I made on the ticket (different storage ports for
each node). For us, it's a great "would really like to have" as our group
will probably need this in production within the next 1 year or less.
However since it hasn't gotten much attention, I'm not sure if we should
add it to the list of "must haves" for 4.0. I'm planning on bringing it up
internally today, as well.

Also, from the previous 4.0 email thread that Jonathan started back in July
(
https://mail-archives.apache.org/mod_mbox/cassandra-dev/201607.mbox/%3CCALdd-zhW3qJ%3DOWida9nMXPj0JOsru7guOYh6-7uTjqEBvacrgQ%40mail.gmail.com%3E
)

- CASSANDRA-5 (thrift removal) - ticket not mentioned explicitly in the
email, but the notion of removing thrift was
- CASSANDRA-10857 (Allow dropping COMPACT STORAGE flag)

Thanks,

-Jason

On Thu, Nov 3, 2016 at 8:43 PM, sankalp kohli 
wrote:

> List looks really good. I will let you know if there is something else we
> plan to add to this list.
>
> On Thu, Nov 3, 2016 at 7:47 PM, Nate McCall  wrote:
>
> > It was brought up recently at the PMC level that our goals as a
> > project are not terribly clear.
> >
> > This is a pretty good point as outside of Jira 'Fix Version' labelling
> > (which we actually suck less at compared to a lot of other ASF
> > projects) this really isnt tracked anywhere outside of general tribal
> > knowledge about who is working on what.
> >
> > I would like to see us change this for two reasons:
> > - it's important we are clear with our community about where we are going
> > - we need to start working more closely together
> >
> > To that end, i've put together a list (in no particular order) of the
> > *major* features in which I know folks are interested, have patches
> > coming, are awaiting design review, etc.:
> >
> > - CASSANDRA-9425 Immutable node-local schema
> > - CASSANDRA-10699 Strongly consistent schema alterations
> > - CASSANDRA-12229 NIO streaming
> > - CASSANDRA-8457 NIO messaging
> > - CASSANDRA-12345 Gossip 2.0
> > - CASSANDRA-9754 Birch trees
> >
> > What did I miss? What else would folks like to see? Specifically, this
> > should be "new stuff that could/will break things" given we are upping
> > the major version.
> >
> > To be clear, it's not my intention to set this in stone and then beat
> > people about the head with it. More to have it there to point it at a
> > high level and foster better communication with our users from the
> > perspective of an open source project.
> >
> > Please keep in mind that given everything else going on, I think it's
> > a fantastic idea to keep this list small and spend some time focusing
> > on stability particularly as we transition to a new release process.
> >
> > -Nate
> >
>

Re: Moderation

[Jason Brown]

Gary,

I've just started looking into the moderator component due to this thread;
I admit I did not know about it before (my fault). Yes, I would like to be
added. Apparently, I need to file an INFRA ticket (as per
https://www.apache.org/dev/committers.html#mailing-list-moderators), which
I will do in the next few minutes.

-Jason

On Fri, Nov 4, 2016 at 9:51 AM, Gary Dusbabek  wrote:

> I'm beginning to wonder if I'm the only one with moderator privs. Any other
> committer/PMCs interested?
>
> Sorry, it's a chore to begin with and I've been traveling this week.
>
> Gary.
>
> On Fri, Nov 4, 2016 at 3:47 PM, Chris Mattmann 
> wrote:
>
> > Hi Folks,
> >
> > Kelly Sommers sent a message to dev@cassandra and I'm trying to figure
> > out if it's in moderation.
> >
> > Can the moderators speak up?
> >
> > Cheers,
> > Chris
> >
> >
>

Re: Moderation

[Jason Brown]

Chris,

Yes, I would like to be added, and here sis the ticket I filed:
https://issues.apache.org/jira/browse/INFRA-12858.

Thanks,

-Jason

On Fri, Nov 4, 2016 at 10:10 AM, Chris Mattmann  wrote:

> I have apmail karma and can add moderators.
>
> Jason I can add you - please confirm you would like to be added. Did you
> file the ticket - if so point me to it. If you haven't yet, no worries I
> can still add you. Let me know. Thanks.
>
> On 2016-11-04 09:54 (-0700), Jason Brown  wrote:
> > Gary,
> >
> > I've just started looking into the moderator component due to this
> thread;
> > I admit I did not know about it before (my fault). Yes, I would like to
> be
> > added. Apparently, I need to file an INFRA ticket (as per
> > https://www.apache.org/dev/committers.html#mailing-list-moderators),
> which
> > I will do in the next few minutes.
> >
> > -Jason
> >
> > On Fri, Nov 4, 2016 at 9:51 AM, Gary Dusbabek 
> wrote:
> >
> > > I'm beginning to wonder if I'm the only one with moderator privs. Any
> other
> > > committer/PMCs interested?
> > >
> > > Sorry, it's a chore to begin with and I've been traveling this week.
> > >
> > > Gary.
> > >
> > > On Fri, Nov 4, 2016 at 3:47 PM, Chris Mattmann 
> > > wrote:
> > >
> > > > Hi Folks,
> > > >
> > > > Kelly Sommers sent a message to dev@cassandra and I'm trying to
> figure
> > > > out if it's in moderation.
> > > >
> > > > Can the moderators speak up?
> > > >
> > > > Cheers,
> > > > Chris
> > > >
> > > >
> > >
> >
>

Re: Moderation

[Jason Brown]

s/sis/is

On Fri, Nov 4, 2016 at 10:12 AM, Jason Brown  wrote:

> Chris,
>
> Yes, I would like to be added, and here sis the ticket I filed:
> https://issues.apache.org/jira/browse/INFRA-12858.
>
> Thanks,
>
> -Jason
>
> On Fri, Nov 4, 2016 at 10:10 AM, Chris Mattmann 
> wrote:
>
>> I have apmail karma and can add moderators.
>>
>> Jason I can add you - please confirm you would like to be added. Did you
>> file the ticket - if so point me to it. If you haven't yet, no worries I
>> can still add you. Let me know. Thanks.
>>
>> On 2016-11-04 09:54 (-0700), Jason Brown  wrote:
>> > Gary,
>> >
>> > I've just started looking into the moderator component due to this
>> thread;
>> > I admit I did not know about it before (my fault). Yes, I would like to
>> be
>> > added. Apparently, I need to file an INFRA ticket (as per
>> > https://www.apache.org/dev/committers.html#mailing-list-moderators),
>> which
>> > I will do in the next few minutes.
>> >
>> > -Jason
>> >
>> > On Fri, Nov 4, 2016 at 9:51 AM, Gary Dusbabek 
>> wrote:
>> >
>> > > I'm beginning to wonder if I'm the only one with moderator privs. Any
>> other
>> > > committer/PMCs interested?
>> > >
>> > > Sorry, it's a chore to begin with and I've been traveling this week.
>> > >
>> > > Gary.
>> > >
>> > > On Fri, Nov 4, 2016 at 3:47 PM, Chris Mattmann 
>> > > wrote:
>> > >
>> > > > Hi Folks,
>> > > >
>> > > > Kelly Sommers sent a message to dev@cassandra and I'm trying to
>> figure
>> > > > out if it's in moderation.
>> > > >
>> > > > Can the moderators speak up?
>> > > >
>> > > > Cheers,
>> > > > Chris
>> > > >
>> > > >
>> > >
>> >
>>
>
>

Re: Rough roadmap for 4.0

[Jason Brown]

+1 to epaxos

On Fri, Nov 4, 2016 at 11:40 AM, Jonathan Haddad  wrote:

> epaxos would be nice, it's been about 2 years since it was started, and
> Blake asked for a first review well over a year ago.  the benchmarks and
> thorough test suite look like a huge improvement over the current
> situation.
>
> https://issues.apache.org/jira/browse/CASSANDRA-6246
>
> On Fri, Nov 4, 2016 at 3:28 AM Edward Capriolo 
> wrote:
>
> > I would like to propose features around seeds:
> > https://issues.apache.org/jira/browse/CASSANDRA-12627
> >
> > I have other follow up issues like getting seeds from Amazon API, or from
> > JNDI/ DNS, etc.
> >
> > I was hoping 12627 was an easy way to grease the wheels.
> >
> >
> > On Fri, Nov 4, 2016 at 8:39 AM, Jason Brown 
> wrote:
> >
> > > Hey Nate,
> > >
> > > I'd like to add CASSANDRA-11559 (Enhance node representation) to the
> > list,
> > > including the comment I made on the ticket (different storage ports for
> > > each node). For us, it's a great "would really like to have" as our
> group
> > > will probably need this in production within the next 1 year or less.
> > > However since it hasn't gotten much attention, I'm not sure if we
> should
> > > add it to the list of "must haves" for 4.0. I'm planning on bringing it
> > up
> > > internally today, as well.
> > >
> > > Also, from the previous 4.0 email thread that Jonathan started back in
> > July
> > > (
> > > https://mail-archives.apache.org/mod_mbox/cassandra-dev/
> > > 201607.mbox/%3CCALdd-zhW3qJ%3DOWida9nMXPj0JOsru7guOYh6-
> > > 7uTjqEBvacrgQ%40mail.gmail.com%3E
> > > )
> > >
> > > - CASSANDRA-5 (thrift removal) - ticket not mentioned explicitly in
> > the
> > > email, but the notion of removing thrift was
> > > - CASSANDRA-10857 (Allow dropping COMPACT STORAGE flag)
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > > On Thu, Nov 3, 2016 at 8:43 PM, sankalp kohli 
> > > wrote:
> > >
> > > > List looks really good. I will let you know if there is something
> else
> > we
> > > > plan to add to this list.
> > > >
> > > > On Thu, Nov 3, 2016 at 7:47 PM, Nate McCall 
> > wrote:
> > > >
> > > > > It was brought up recently at the PMC level that our goals as a
> > > > > project are not terribly clear.
> > > > >
> > > > > This is a pretty good point as outside of Jira 'Fix Version'
> > labelling
> > > > > (which we actually suck less at compared to a lot of other ASF
> > > > > projects) this really isnt tracked anywhere outside of general
> tribal
> > > > > knowledge about who is working on what.
> > > > >
> > > > > I would like to see us change this for two reasons:
> > > > > - it's important we are clear with our community about where we are
> > > going
> > > > > - we need to start working more closely together
> > > > >
> > > > > To that end, i've put together a list (in no particular order) of
> the
> > > > > *major* features in which I know folks are interested, have patches
> > > > > coming, are awaiting design review, etc.:
> > > > >
> > > > > - CASSANDRA-9425 Immutable node-local schema
> > > > > - CASSANDRA-10699 Strongly consistent schema alterations
> > > > > - CASSANDRA-12229 NIO streaming
> > > > > - CASSANDRA-8457 NIO messaging
> > > > > - CASSANDRA-12345 Gossip 2.0
> > > > > - CASSANDRA-9754 Birch trees
> > > > >
> > > > > What did I miss? What else would folks like to see? Specifically,
> > this
> > > > > should be "new stuff that could/will break things" given we are
> > upping
> > > > > the major version.
> > > > >
> > > > > To be clear, it's not my intention to set this in stone and then
> beat
> > > > > people about the head with it. More to have it there to point it
> at a
> > > > > high level and foster better communication with our users from the
> > > > > perspective of an open source project.
> > > > >
> > > > > Please keep in mind that given everything else going on, I think
> it's
> > > > > a fantastic idea to keep this list small and spend some time
> focusing
> > > > > on stability particularly as we transition to a new release
> process.
> > > > >
> > > > > -Nate
> > > > >
> > > >
> > >
> >
>

Re: [VOTE] Close client-...@cassandra.apache.org mailing list

[Jason Brown]

+1

On Sunday, November 6, 2016, Gary Dusbabek  wrote:

> +1 let's close it.
>
> Gary.
>
>
> On Mon, Nov 7, 2016 at 5:11 AM, Jeff Jirsa  > wrote:
>
> > There exists a nearly unused mailing list,
> client-...@cassandra.apache.org 
> > [0].
> >
> > This is a summary of the email threads over the past 12 months on that
> > list:
> >
> > 1) ApacheCon Seville CFP Close notice
> > 2) Datastax .NET driver question
> > 3) Datastax Java driver question
> > 4) FOSDEM announce
> > 5) ApacheCon NA CFP Open noticed
> >
> > In order to avoid confusion, and given the lack of relevant and
> > appropriate traffic, I propose we close the client-dev@ list entirely.
> > Any traffic appropriate for the client-dev@ list would likely be better
> > served if it were directed at dev@, which is more active.
> >
> > This vote will remain open for 72 hours.
> >
> > 0: https://lists.apache.org/list.html?client-...@cassandra.apache.org
> >
> >
> >
>

Re: New 'cassandra-builds' git repo, or in-tree?

[Jason Brown]

+1, even though I think you've already done it ;) Thanks!

On Fri, Nov 11, 2016 at 1:51 PM, Michael Shuler 
wrote:

> On 11/11/2016 01:02 PM, Nate McCall wrote:
> >> It may also allow a
> >> lower barrier for contributors interested in helping with specifically
> >> build/test infrastructure.
> >
> > Good point for new repo.
>
> Requested a new 'cassandra-builds' git repo! Thanks.
>
> --
> Michael
>
>

Re: Rough roadmap for 4.0

[Jason Brown]

Jason,

That's a separate topic, but we will have a different vote on how the
branching/release strategy should be for the future.

On Thursday, November 17, 2016, jason zhao yang 
wrote:

> Hi,
>
> Will we still use tick-tock release for 4.x and 4.0.x ?
>
> Stefan Podkowinski >于2016年11月16日周三
> 下午4:52写道：
>
> > From my understanding, this will also effect EOL dates of other branches.
> >
> > "We will maintain the 2.2 stability series until 4.0 is released, and 3.0
> > for six months after that.".
> >
> >
> > On Wed, Nov 16, 2016 at 5:34 AM, Nate McCall  > wrote:
> >
> > > Agreed. As long as we have a goal I don't see why we have to adhere to
> > > arbitrary date for 4.0.
> > >
> > > On Nov 16, 2016 1:45 PM, "Aleksey Yeschenko"  >
> > wrote:
> > >
> > > > I’ll comment on the broader issue, but right now I want to elaborate
> on
> > > > 3.11/January/arbitrary cutoff date.
> > > >
> > > > Doesn’t matter what the original plan was. We should continue with
> 3.X
> > > > until all the 4.0 blockers have been
> > > > committed - and there are quite a few of them remaining yet.
> > > >
> > > > So given all the holidays, and the tickets remaining, I’ll personally
> > be
> > > > surprised if 4.0 comes out before
> > > > February/March and 3.13/3.14. Nor do I think it’s an issue.
> > > >
> > > > —
> > > > AY
> > > >
> > > > On 16 November 2016 at 00:39:03, Mick Semb Wever (
> > m...@thelastpickle.com 
> > > )
> > > > wrote:
> > > >
> > > > On 4 November 2016 at 13:47, Nate McCall  > wrote:
> > > >
> > > > > Specifically, this should be "new stuff that could/will break
> things"
> > > > > given we are upping
> > > > > the major version.
> > > > >
> > > >
> > > >
> > > > How does this co-ordinate with the tick-tock versioning¹ leading up
> to
> > > the
> > > > 4.0 release?
> > > >
> > > > To just stop tick-tock and then say yeehaa let's jam in all the
> > breaking
> > > > changes we really want seems to be throwing away some of the learnt
> > > wisdom,
> > > > and not doing a very sane transition from tick-tock to
> > > > features/testing/stable². I really hope all this is done in a way
> that
> > > > continues us down the path towards a stable-master.
> > > >
> > > > For example, are we fixing the release of 4.0 to November? or
> > continuing
> > > > tick-tocks until we complete the 4.0 roadmap? or starting the
> > > > features/testing/stable branching approach with 3.11?
> > > >
> > > >
> > > > Background:
> > > > ¹) Sylvain wrote in an earlier thread titled "A Home for 4.0"
> > > >
> > > > > And as 4.0 was initially supposed to come after 3.11, which is
> > coming,
> > > > it's probably time to have a home for those tickets.
> > > >
> > > > ²) The new versioning scheme slated for 4.0, per the "Proposal -
> 3.5.1"
> > > > thread
> > > >
> > > > > three branch plan with “features”, “testing”, and “stable” starting
> > > with
> > > > 4.0?
> > > >
> > > >
> > > > Mick
> > > >
> > >
> >
>

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

[Jason Brown]

Hey all,

Here's an update on the following items:

NIO meassing/streaming - first is being reviewed; second is getting close
to review time
Gossip 2.0 - TL;DR I don't plan on moving cluster metadata (the current
"gossip" data) onto the new gossip/membership stack until 5.0, so it's not
a 4.0 blocker. I'll update #12345 with the details I'm thinking about. I
still want to start getting this code in, though, or at least in discussion.
Birch - on track
#11559 (enhanced node representation) - decided it's *not* something we
need wrt #7544 storage port configurable per node, so we are punting on
#11559
#6246 epaxos - if we're targeting Q1 2017 for 4.0, we probably can't get it
ready by then
#7544 storage port configurable per node - on track

So basically, I've removed two items off that list of blockers for 4.0.
Hope that helps

-Jason



On Fri, Nov 18, 2016 at 9:25 AM, sankalp kohli 
wrote:

> Hi Nate,
>  Most of the JIRAs in the middle are being rebased or being
> reviewed and code is already out there. These will make 4.0 a very solid
> release.
>
> Thanks,
> Sankalp
>
> On Thu, Nov 17, 2016 at 5:10 PM, Ben Bromhead  wrote:
>
> > We are happy to start testing against completed features. Ideally once
> > everything is ready for an RC (to catch interaction bugs), but we can do
> > sooner for features where it make sense and are finished earlier.
> >
> > On Thu, 17 Nov 2016 at 16:47 Nate McCall  wrote:
> >
> > > To sum up that other thread (I very much appreciate everyone's input,
> > > btw), here is an aggregate list of large, breaking 4.0 proposed
> > > changes:
> > >
> > > CASSANDRA-9425 Immutable node-local schema
> > > CASSANDRA-10699 Strongly consistent schema alterations
> > > --
> > > CASSANDRA-12229 NIO streaming
> > > CASSANDRA-8457 NIO messaging
> > > CASSANDRA-12345 Gossip 2.0
> > > CASSANDRA-9754 Birch trees
> > > CASSANDRA-11559 enhanced node representation
> > > CASSANDRA-6246 epaxos
> > > CASSANDRA-7544 storage port configurable per node
> > > --
> > > CASSANDRA-5 remove thrift support
> > > CASSANDRA-10857 dropping compact storage
> > >
> > > Again, this is the "big things that will probably break stuff" list
> > > and thus should happen with a major (did I miss anything?). There
> > > were/are/will be other smaller issues, but we don't really need to
> > > keep them in front of us for this discussion as they can/will just
> > > kind of happen w/o necessarily affecting anything else.
> > >
> > > That all said, since we are 'doing a software' we need to start
> > > thinking about the above in balance with resources and time. However,
> > > a lot of the above items do have a substantial amount of code written
> > > against them so it's not as daunting as it seems.
> > >
> > > What I would like us to discuss is rough timelines and what is needed
> > > to get these out the door.
> > >
> > > One thing that sticks out to me: that big chunk in the middle there is
> > > coming out of the same shop in Cupertino. I'm nervous about that. Not
> > > that that ya'll are not capable, I'm solely looking at it from the
> > > "that is a big list of some pretty hard shit" perspective.
> > >
> > > So what else do we need to discuss to get these completed? How and
> > > where can other folks pitch in?
> > >
> > > -Nate
> > >
> > --
> > Ben Bromhead
> > CTO | Instaclustr 
> > +1 650 284 9692
> > Managed Cassandra / Spark on AWS, Azure and Softlayer
> >
>

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

[Jason Brown]

should assume that we’re ditching tick/tock. I’ll post a thread on
> >>>>> 4.0-and-beyond here in a few minutes.
> >>>>>
> >>>>> The advantage of a prod release every 6 months is fewer incentive to
> >> push
> >>>>> unfinished work into a release.
> >>>>> The disadvantage of a prod release every 6 months is then we either
> >> have
> >>>>> a very short lifespan per-release, or we have to maintain lots of
> >> active
> >>>>> releases.
> >>>>>
> >>>>> 2.1 has been out for over 2 years, and a lot of people (including us)
> >> are
> >>>>> running it in prod – if we have a release every 6 months, that means
> >> we’d
> >>>>> be supporting 4+ releases at a time, just to keep parity with what we
> >> have
> >>>>> now? Maybe that’s ok, if we’re very selective about ‘support’ for 2+
> >> year
> >>>>> old branches.
> >>>>>
> >>>>>
> >>>>> On 11/18/16, 3:10 PM, "beggles...@apple.com <;> on behalf
> >> of Blake
> >>>>> Eggleston" > wrote:
> >>>>>
> >>>>>>> While stability is important if we push back large "core" changes
> >>>>> until later we're just setting ourselves up to face the same issues
> >> later on
> >>>>>>
> >>>>>> In theory, yes. In practice, when incomplete features are earmarked
> >> for
> >>>>> a certain release, those features are often rushed out, and not
> always
> >>>>> fully baked.
> >>>>>>
> >>>>>> In any case, I don’t think it makes sense to spend too much time
> >>>>> planning what goes into 4.0, and what goes into the next major
> release
> >> with
> >>>>> so many release strategy related decisions still up in the air. Are
> we
> >>>>> going to ditch tick-tock? If so, what will it’s replacement look
> like?
> >>>>> Specifically, when will the next “production” release happen? Without
> >>>>> knowing that, it's hard to say if something should go in 4.0, or 4.5,
> >> or
> >>>>> 5.0, or whatever.
> >>>>>>
> >>>>>> The reason I suggested a production release every 6 months is
> because
> >>>>> (in my mind) it’s frequent enough that people won’t be tempted to
> rush
> >>>>> features to hit a given release, but not so frequent that it’s not
> >>>>> practical to support. It wouldn’t be the end of the world if some of
> >> these
> >>>>> tickets didn’t make it into 4.0, because 4.5 would fine.
> >>>>>>
> >>>>>> On November 18, 2016 at 1:57:21 PM, kurt Greaves (
> >> k...@instaclustr.com <;>)
> >>>>> wrote:
> >>>>>>
> >>>>>>> On 18 November 2016 at 18:25, Jason Brown  >> <;>> wrote:
> >>>>>>>
> >>>>>>> #11559 (enhanced node representation) - decided it's *not*
> something
> >> we
> >>>>>>> need wrt #7544 storage port configurable per node, so we are
> punting
> >> on
> >>>>>>>
> >>>>>>
> >>>>>> #12344 - Forward writes to replacement node with same address during
> >>>>> replace
> >>>>>> depends on #11559. To be honest I'd say #12344 is pretty important,
> >>>>>> otherwise it makes it difficult to replace nodes without potentially
> >>>>>> requiring client code/configuration changes. It would be nice to get
> >>>>> #12344
> >>>>>> in for 4.0. It's marked as an improvement but I'd consider it a bug
> >> and
> >>>>>> thus think it could be included in a later minor release.
> >>>>>>
> >>>>>> Introducing all of these in a single release seems pretty risky. I
> >> think
> >>>>> it
> >>>>>>> would be safer to spread these out over a few 4.x releases (as
> >> they’re
> >>>>>>> finished) and give them time to stabilize before including them in
> an
> >>>>> LTS
> >>>>>>> release. The downside would be having to maintain backwards
> >>>>> compatibility
> >>>>>>> across the 4.x versions, but that seems preferable to delaying the
> >>>>> release
> >>>>>>> of 4.0 to include these, and having another big bang release.
> >>>>>>
> >>>>>>
> >>>>>> I don't think anyone expects 4.0.0 to be stable. It's a major
> version
> >>>>>> change with lots of new features; in the production world people
> don't
> >>>>>> normally move to a new major version until it has been out for quite
> >> some
> >>>>>> time and several minor releases have passed. Really, most people are
> >> only
> >>>>>> migrating to 3.0.x now. While stability is important if we push back
> >>>>> large
> >>>>>> "core" changes until later we're just setting ourselves up to face
> the
> >>>>> same
> >>>>>> issues later on. There should be enough uptake on the early releases
> >> of
> >>>>> 4.0
> >>>>>> from new users to help test and get it to a production-ready state.
> >>>>>>
> >>>>>>
> >>>>>> Kurt Greaves
> >>>>>> k...@instaclustr.com <;>
> >>>>>
> >>>>>
> >>>> I don't think anyone expects 4.0.0 to be stable
> >>>>
> >>>> Someone previously described 3.0 as the "break everything release".
> >>>>
> >>>> We know that many people are still 2.1 and 3.0. Cassandra will always
> be
> >>>> maintaining 3 or 4 active branches and have adoption issues if
> releases
> >> are
> >>>> not stable and usable.
> >>>>
> >>>> Being that cassandra was 1.0 years ago I expect things to be stable.
> >> Half
> >>>> working features , or added this broke that are not appealing to me.
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Sorry this was sent from mobile. Will do less grammar and spell check
> >> than
> >>>> usual.
> >>>>
> >>>
> >>>
> >>> --
> >>> Sorry this was sent from mobile. Will do less grammar and spell check
> >> than
> >>> usual.
> >>
> >
> >
> > --
> > Sorry this was sent from mobile. Will do less grammar and spell check
> than
> > usual.
>

Re: Summary of 4.0 Large Features/Breaking Changes (Was: Rough roadmap for 4.0)

[Jason Brown]

+1 to everything Blake said. Bonus points for property-based state testing
(a la ScalaCheck/QuickCheck).

This being said, are we drifting from the original topic of this thread?
Should we decide features for 4.0? It seems to me testing concerns might be
for another thread? Is it worthwhile voting on features without a clearer
testing/release roadmap/plan?

Thanks,

Jason

On Sun, Nov 20, 2016 at 11:23 Blake Eggleston  wrote:

> > I'm not sure how the apache team does this. Perhaps individual engineers
> can run some modern version at a company of theirs, altho that seems
> unlikely, but as an Apache org, i just don't see how that happens.
>
> > To me it seems like the Apache Cassandra infrastructure itself needs to
> stand up a multinode live instance running some 'real-world' example
> that is getting pounded, so that we can stage feature branches to really
> test them.
>
> Not having access to test hardware as an apache org is a problem, but
> there’s also a lot of room for improvement on the junit testing and
> testability side of things. That’s true for both local and distributed
> components, but more junit coverage of the distributed mechanisms would
> make not having test hardware suck less. With distributed algorithms (like
> gossip 2.0) one of the limitations of testing with live nodes is that
> you’re often just testing the happy path. Reliably and repeatably testing
> how the system responds to weird edge cases involving specific ordering of
> events across nodes is very difficult to do.
>
> I’d written epaxos with this sort of testing in mind, and was able to do a
> lot of testing of obscure failure scenarios (see
> https://github.com/bdeggleston/cassandra/blob/CASSANDRA-6246-trunk/test/unit/org/apache/cassandra/service/epaxos/integration/EpaxosIntegrationRF3Test.java#L144
>  for
> an example). This doesn’t obviate the need to test on real clusters of
> course, but it does increase confidence that the system will behave
> correctly under load, and reduce the amount of things you’re relying on a
> loaded test cluster to reveal.
>
> On November 20, 2016 at 9:02:55 AM, Dave Brosius (dbros...@mebigfatguy.com)
> wrote:
>
> >> We fully intend to "engineer and test the snot out of" the changes
> we are working on as the whole point of us working on them is so we
> *can* run them in production, at our scale.
>
> I'm not sure how the apache team does this. Perhaps individual engineers
> can run some modern version at a company of theirs, altho that seems
> unlikely, but as an Apache org, i just don't see how that happens.
>
> To me it seems like the Apache Cassandra infrastructure itself needs to
> stand up a multinode live instance running some 'real-world' example
> that is getting pounded, so that we can stage feature branches to really
> test them.
>
> Otherwise we will forever be basing versions on the poor test saps who
> decide they are willing to risk all to upgrade to the cutting edge, and
> why, everyone believes in the adage, don't upgrade until at least .6
>
> --dave
>
>
> On 11/20/2016 09:50 AM, Jason Brown wrote:
> > Hey all,
> >
> > One of the goals on my team, when working on large patches, is to get
> > community feedback on these initiatives before throwing them into prod.
> > This gets us a wider net of feedback (see Sylvain's continuing excellent
> > rounds of feedback to my work on CASSANDRA-8457), as well as making sure
> we
> > don't go too far off the deep end in terms of straying from the community
> > version. The latter point is crucial because if we make too many
> > incompatible changes to, for example, the internode messaging protocol or
> > the CQL protocol or the sstable file format, and deploy that, it may be
> > very difficult, if not impossible, to rectify with future, in-development
> > versions of cassandra.
> >
> > We fully intend to "engineer and test the snot out of" the changes we are
> > working on as the whole point of us working on them is so we *can* run
> them
> > in production, at our scale. We aren't expecting others in the community
> to
> > dog food it for us. There will be a delay between committing something
> > upstream, and us backporting it to a current version we run in production
> > and actually deploying it. However, you can be sure that any bugs we find
> > will be fixed ASAP; we have many users counting on it.
> >
> > Thanks for listening,
> >
> > -Jason
> >
> >
> > On Sat, Nov 19, 2016 at 11:04 AM, Blake Eggleston 
> > wrote:
> >
> >> I think Ed's just using gossip 2.0 as a hypothetical example. His p

Re: [VOTE] 3.X branch feature freeze

[Jason Brown]

+1

On Fri, Jan 13, 2017 at 10:16 AM, Benjamin Roth 
wrote:

> Progress:
> Yes and no. I made a patch that made our cluster stable performance wise
> but introduces a consistency issue I am aware of. We can deal with it and I
> prefer this over severe performance problems. But this is nothing you can
> offer to regular users. I created a bunch of tickets related to this.
> Making bootstrap + decommision performant + consistent should not be much
> effort.
> Making repairs performant + consistent + fix incremental repairs will be
> probably more effort. I wanted to investigate more before xmas but did not
> find the time until now. It is on my agenda and I appreciate any support.
>
> Tickets I created recently:
> https://issues.apache.org/jira/browse/CASSANDRA-13073
> https://issues.apache.org/jira/browse/CASSANDRA-13066
> https://issues.apache.org/jira/browse/CASSANDRA-13065
> https://issues.apache.org/jira/browse/CASSANDRA-13064
>
> Also important:
> https://issues.apache.org/jira/browse/CASSANDRA-12888
>
>
> 2017-01-13 18:55 GMT+01:00 Jonathan Haddad :
>
> > +1 (non binding) to feature freeze.
> >
> > I also like the idea of stabilizing MVs.  Ben, you've probably been the
> > most vocal about the issues, have you made any progress towards making
> them
> > work any better during bootstrap / etc?  Any idea of fixing them is a
> major
> > undertaking?
> >
> > Jon
> >
> > On Fri, Jan 13, 2017 at 9:39 AM Benjamin Roth 
> > wrote:
> >
> > +1 also I appreciate any effort on MV stability. It is an official 3.x
> > feature but not production ready for the masses.
> >
> > Am 13.01.2017 18:34 schrieb "Jonathan Ellis" :
> >
> > > +1
> > >
> > > On Fri, Jan 13, 2017 at 11:21 AM, Aleksey Yeschenko <
> alek...@apache.org>
> > > wrote:
> > >
> > > > Hi all!
> > > >
> > > > It seems like we have a general consensus on ending tick-tock at
> 3.11,
> > > and
> > > > moving
> > > > on to stabilisation-only for 3.11.x series.
> > > >
> > > > In light of this, I suggest immediate feature freeze in the 3.X
> branch.
> > > >
> > > > Meaning that only bug fixes go to the 3.11/3.X branch from now on.
> > > >
> > > > All new features that haven’t be committed yet should go to trunk
> only
> > > > (4.0), if the vote passes.
> > > >
> > > > What do you think?
> > > >
> > > > Thanks.
> > > >
> > > > --
> > > > AY
> > >
> > >
> > >
> > >
> > > --
> > > Jonathan Ellis
> > > co-founder, http://www.datastax.com
> > > @spyced
> > >
> >
>
>
>
> --
> Benjamin Roth
> Prokurist
>
> Jaumo GmbH · www.jaumo.com
> Wehrstraße 46 · 73035 Göppingen · Germany
> Phone +49 7161 304880-6 · Fax +49 7161 304880-1
> AG Ulm · HRB 731058 · Managing Director: Jens Kammerer
>

Re: Wrapping up tick-tock

[Jason Brown]

It's fine to limit the minimum time between major releases to six months,
but I do not think we should force a major just because n months have
passed. I think we should up the major only when we have significant
(possibly breaking) changes/features. It would seem odd to have a 6.0
that's basically the same as 4.0 (in terms of features and protocol/format
compatibility).

Thoughts?

On Wed, Jan 11, 2017 at 1:58 AM, Stefan Podkowinski 
wrote:

> I honestly don't understand the release cadence discussion. The 3.x branch
> is far from production ready. Is this really the time to plan the next
> major feature releases on top of it, instead of focusing to stabilize 3.x
> first? Who knows how long that would take, even if everyone would
> exclusively work on bug fixing (which I think should happen).
>
> On Tue, Jan 10, 2017 at 4:29 PM, Jonathan Haddad 
> wrote:
>
> > I don't see why it has to be one extreme (yearly) or another (monthly).
> > When you had originally proposed Tick Tock, you wrote:
> >
> > "The primary goal is to improve release quality.  Our current major “dot
> > zero” releases require another five or six months to make them stable
> > enough for production.  This is directly related to how we pile features
> in
> > for 9 to 12 months and release all at once.  The interactions between the
> > new features are complex and not always obvious.  2.1 was no exception,
> > despite DataStax hiring a full tme test engineering team specifically for
> > Apache Cassandra."
> >
> > I agreed with you at the time that the yearly cycle was too long to be
> > adding features before cutting a release, and still do now.  Instead of
> > elastic banding all the way back to a process which wasn't working
> before,
> > why not try somewhere in the middle?  A release every 6 months (with
> > monthly bug fixes for a year) gives:
> >
> > 1. long enough time to stabilize (1 year vs 1 month)
> > 2. not so long things sit around untested forever
> > 3. only 2 releases (current and previous) to do bug fix support at any
> > given time.
> >
> > Jon
> >
> > On Tue, Jan 10, 2017 at 6:56 AM Jonathan Ellis 
> wrote:
> >
> > > Hi all,
> > >
> > > We’ve had a few threads now about the successes and failures of the
> > > tick-tock release process and what to do to replace it, but they all
> died
> > > out without reaching a robust consensus.
> > >
> > > In those threads we saw several reasonable options proposed, but from
> my
> > > perspective they all operated in a kind of theoretical fantasy land of
> > > testing and development resources.  In particular, it takes around a
> > > person-week of effort to verify that a release is ready.  That is,
> going
> > > through all the test suites, inspecting and re-running failing tests to
> > see
> > > if there is a product problem or a flaky test.
> > >
> > > (I agree that in a perfect world this wouldn’t be necessary because
> your
> > > test ci is always green, but see my previous framing of the perfect
> world
> > > as a fantasy land.  It’s also worth noting that this is a common
> problem
> > > for large OSS projects, not necessarily something to beat ourselves up
> > > over, but in any case, that's our reality right now.)
> > >
> > > I submit that any process that assumes a monthly release cadence is not
> > > realistic from a resourcing standpoint for this validation.  Notably,
> we
> > > have struggled to marshal this for 3.10 for two months now.
> > >
> > > Therefore, I suggest first that we collectively roll up our sleeves to
> > vet
> > > 3.10 as the last tick-tock release.  Stick a fork in it, it’s done.  No
> > > more tick-tock.
> > >
> > > I further suggest that in place of tick tock we go back to our old
> model
> > of
> > > yearly-ish releases with as-needed bug fix releases on stable branches,
> > > probably bi-monthly.  This amortizes the release validation problem
> over
> > a
> > > longer development period.  And of course we remain free to ramp back
> up
> > to
> > > the more rapid cadence envisioned by the other proposals if we increase
> > our
> > > pool of QA effort or we are able to eliminate flakey tests to the point
> > > that a long validation process becomes unnecessary.
> > >
> > > (While a longer dev period could mean a correspondingly more painful
> test
> > > validation process at the end, my experience is that most of the
> > validation
> > > cost is “fixed” in the form of flaky tests and thus does not increase
> > > proportionally to development time.)
> > >
> > > Thoughts?
> > >
> > > --
> > > Jonathan Ellis
> > > co-founder, http://www.datastax.com
> > > @spyced
> > >
> >
>

Re: Instaclustr Masters scholarship

[Jason Brown]

Ben,

This is pretty cool! Almost wish I could apply ;)

>> "Distributed database systems are relatively new technology"
https://www.ietf.org/rfc/rfc677.txt.pdf - lol, 1975 :)

-Jason

On Tue, Feb 7, 2017 at 7:26 PM, Ben Bromhead  wrote:

> As part of our commitment to contributing back to the Apache Cassandra open
> source project and the wider community we are always looking for ways we
> can foster knowledge sharing and improve usability of Cassandra itself. One
> of the ways we have done so previously was to open up our internal builds
> and versions of Cassandra (https://github.com/instaclustr/cassandra).
>
> We have also been looking at a few novel or outside the box ways we can
> further contribute back to the community. As such, we are sponsoring a
> masters project in conjunction with the Australian based University of
> Canberra. Instaclustr’s staff will be available to provide advice and
> feedback to the successful candidate.
>
> *Scope*
> Distributed database systems are relatively new technology compared to
> traditional relational databases. Distributed advantages provide
> significant advantages in terms of reliability and scalability but often at
> a cost of increased complexity. This complexity presents challenges for
> testing of these systems to prove correct operation across all possible
> system states. The scope of this masters scholarship is to use the Apache
> Cassandra repair process as an example to consider and improve available
> approaches to distributed database systems testing.
>
> The repair process in Cassandra is a scheduled process that runs to ensure
> the multiple copies of each piece of data that is maintained by Cassandra
> are kept synchronised. Correct operation of repairs has been an ongoing
> challenge for the Cassandra project partly due to the difficulty in
> designing and developing  comprehensive automated tests for this
> functionality.
>
> The expected scope of this project is to:
>
>- survey and understand the existing testing framework available as part
>of the Cassandra project, particularly as it pertains to testing repairs
>- consider, research and develop enhanced approaches to testing of
>repairs
>- submit any successful approaches to the Apache Cassandra project for
>feedback and inclusion in the project code base
>
> Australia is a pretty great place to advance your education and is
> welcoming of foreign students.
>
> We are also open to sponsoring a PhD project with a more in depth focus for
> the right candidate.
>
> For more details please don't hesitate to get in touch with myself or reach
> out to i...@instaclustr.com.
>
> Cheers
>
> Ben
> --
> Ben Bromhead
> CTO | Instaclustr 
> +1 650 284 9692
> Managed Cassandra / Spark on AWS, Azure and Softlayer
>

New committers announcement

[Jason Brown]

Hello all,

It's raining new committers here in Apache Cassandra!  I'd like to announce
the following individuals are now committers for the project:

Branimir Lambov
Paulo Motta
Stefan Pokowinski
Ariel Weisberg
Blake Eggleston
Alex Petrov
Joel Knighton

Congratulations all! Please keep the excellent contributions coming.

Thanks,

-Jason Brown

Re: New committers announcement

[Jason Brown]

.. and I misspelled Stefan's last name :(. It should read "Stefan
Podkowinski". All apologies, Stefan.

On Tue, Feb 14, 2017 at 1:31 PM, Jason Brown  wrote:

> Hello all,
>
> It's raining new committers here in Apache Cassandra!  I'd like to
> announce the following individuals are now committers for the project:
>
> Branimir Lambov
> Paulo Motta
> Stefan Pokowinski
> Ariel Weisberg
> Blake Eggleston
> Alex Petrov
> Joel Knighton
>
> Congratulations all! Please keep the excellent contributions coming.
>
> Thanks,
>
> -Jason Brown
>

Re: [VOTE] Release Apache Cassandra 2.1.17

[Jason Brown]

+1

On Wed, Feb 15, 2017 at 5:16 PM, Michael Shuler 
wrote:

> I propose the following artifacts for release as 2.1.17.
>
> sha1: 943db2488c8b62e1fbe03b132102f0e579c9ae17
> Git:
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/2.1.17-tentative
> Artifacts:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1140/org/apache/cassandra/apache-cassandra/2.1.17/
> Staging repository:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1140/
>
> The Debian packages are available here: http://people.apache.org/~mshuler
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: (CHANGES.txt) https://goo.gl/17RivH
> [2]: (NEWS.txt) https://goo.gl/axKXys
>
> --
> Kind regards,
> Michael Shuler
>
>

Re: [VOTE] Release Apache Cassandra 2.2.9

[Jason Brown]

+1

On Wed, Feb 15, 2017 at 5:16 PM, Michael Shuler 
wrote:

> I propose the following artifacts for release as 2.2.9.
>
> sha1: 70a08f1c35091a36f7d9cc4816259210c2185267
> Git:
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/2.2.9-tentative
> Artifacts:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1139/org/apache/cassandra/apache-cassandra/2.2.9/
> Staging repository:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1139/
>
> The Debian packages are available here: http://people.apache.org/~mshuler
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: (CHANGES.txt) https://goo.gl/AYblr5
> [2]: (NEWS.txt) https://goo.gl/gIXxgR
>
> --
> Kind regards,
> Michael Shuler
>
>

Re: [VOTE] Release Apache Cassandra 3.0.11

[Jason Brown]

+1

On Wed, Feb 15, 2017 at 5:15 PM, Michael Shuler 
wrote:

> I propose the following artifacts for release as 3.0.11.
>
> sha1: 338226e042a22242645ab54a372c7c1459e78a01
> Git:
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.0.11-tentative
> Artifacts:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1138/org/apache/cassandra/apache-cassandra/3.0.11/
> Staging repository:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1138/
>
> The Debian packages are available here: http://people.apache.org/~mshuler
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: (CHANGES.txt) https://goo.gl/ztSHQJ
> [2]: (NEWS.txt) https://goo.gl/nrengr
>
> --
> Kind regards,
> Michael Shuler
>
>

Re: Upgrade the jna version to 4.3.0

[Jason Brown]

Hi Amit,

Can you open a Jira for that? Also we can figure out which branches to
target for the upgrade on the Jira.

Thanks,

Jason
On Sun, Mar 5, 2017 at 08:25 Amitkumar Ghatwal  wrote:

>
>
> Hi All,
>
> Could you please upgrade the jna version present in the github cassandra
> location :
> https://github.com/apache/cassandra/blob/trunk/lib/jna-4.0.0.jar
> to below latest version   - 4.3.0 -
>
> http://repo1.maven.org/maven2/net/java/dev/jna/jna/4.3.0/jna-4.3.0-javadoc.jar
>  .
>
> Let me know the process to upgrade the same .
>
> Regards,
> Amit
>
>
>

Re: [VOTE] Release Apache Cassandra 3.0.12

[Jason Brown]

+1

On Wed, Mar 8, 2017 at 5:38 AM, Aleksey Yeschenko 
wrote:

> +1
>
> --
> AY
>
> On 7 March 2017 at 16:15:32, Michael Shuler (mich...@pbandjelly.org)
> wrote:
>
> I propose the following artifacts for release as 3.0.12.
>
> This release addresses a possible 2.1->3.0 upgrade issue[3], along with
> a few fixes committed since 3.0.11.
>
> sha1: 50560aaf0f2d395271ade59ba9b900a84cae70f1
> Git:
> http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=
> shortlog;h=refs/tags/3.0.12-tentative
> Artifacts:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1141/org/apache/cassandra/apache-cassandra/3.0.12/
> Staging repository:
> https://repository.apache.org/content/repositories/
> orgapachecassandra-1141/
>
> The Debian packages are available here: http://people.apache.org/~mshuler
>
> The vote will be open for 72 hours (longer if needed).
>
> [1]: (CHANGES.txt) https://goo.gl/RtBFVA
> [2]: (NEWS.txt) https://goo.gl/GGI0aq
> [3]: https://issues.apache.org/jira/browse/CASSANDRA-13294
>
>

Testing and jira tickets

[Jason Brown]

Hey all,

A nice convention we've stumbled into wrt to patches submitted via Jira is
to post the results of unit test and dtest runs to the ticket (to show the
patch doesn't break things). Many contributors have used the
DataStax-provided cassci system, but that's not the best long term
solution. To that end, I'd like to start a conversation about what is the
best way to proceed going forward, and then add it to the "How to
contribute" docs.

As an example, should contributors/committers run dtests and unit tests on
*some* machine (publicly available or otherwise), and then post those
results to the ticket? This could be a link to a build system, like what we
have with cassci, or just  upload the output of the test run(s).

I don't have any fixed notions, and am looking forward to hearing other's
ideas.

Thanks,

-Jason

p.s. a big thank you to DataStax for providing the cassci system

Re: Testing and jira tickets

[Jason Brown]

Jon and Brandon,

I'd actually like to narrow the discussion, and keep it focused to my
original topic. Those are two excellent topics that should be addressed,
and the solution(s) might be the same or similar as the outcome of this.
However, I feel they deserve their own message thread.

Thanks for understanding,

-Jason

On Thu, Mar 9, 2017 at 11:27 AM, Brandon Williams  wrote:

> Let me further broaden this discussion to include github branches, which
> are often linked on tickets, and then later deleted.  This forces a person
> to search through git to actually see the patch, and that process can be a
> little rough (especially since we all know if you're gonna make a typo,
> it's going to be in the commit, and it's probably going to be the ticket
> number.)
>
> On Thu, Mar 9, 2017 at 1:00 PM, Jonathan Haddad  wrote:
>
> > If you don't mind, I'd like to broaden the discussion a little bit to
> also
> > discuss performance related patches.  For instance, CASSANDRA-13271 was a
> > performance / optimization related patch that included *zero* information
> > on if there was any perf improvement or a regression as a result of the
> > change, even though I've asked twice for that information.
> >
> > In addition to "does this thing break anything" we should be asking "how
> > does this patch affect performance?" (and were the appropriate docs
> > included, but that's another topic altogether)
> >
> > On Thu, Mar 9, 2017 at 10:51 AM Jason Brown 
> wrote:
> >
> > > Hey all,
> > >
> > > A nice convention we've stumbled into wrt to patches submitted via Jira
> > is
> > > to post the results of unit test and dtest runs to the ticket (to show
> > the
> > > patch doesn't break things). Many contributors have used the
> > > DataStax-provided cassci system, but that's not the best long term
> > > solution. To that end, I'd like to start a conversation about what is
> the
> > > best way to proceed going forward, and then add it to the "How to
> > > contribute" docs.
> > >
> > > As an example, should contributors/committers run dtests and unit tests
> > on
> > > *some* machine (publicly available or otherwise), and then post those
> > > results to the ticket? This could be a link to a build system, like
> what
> > we
> > > have with cassci, or just  upload the output of the test run(s).
> > >
> > > I don't have any fixed notions, and am looking forward to hearing
> other's
> > > ideas.
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > > p.s. a big thank you to DataStax for providing the cassci system
> > >
> >
>

Re: Testing and jira tickets

[Jason Brown]

To Ariel's point, I don't think we can expect all contributors to run all
utesss/dtests, especially when the patch spans multiple branches. On that
front, I, like Ariel and many others, typically create our own branch of
the patch and have executed the tests. I think this is a reasonable system,
if slightly burdensome, given our project and our needs/demands on it.

Whoever runs the test, is it reasonable to expect the results to become
available on the ticket? As the CI is moving to the Apache infrastructure,
that will be the final arbiter of " patch works or breaks things", but I
suspect executing the tests anywhere else will still be a very good
indicator of the viability of the patch.

Thoughts?

On Thu, Mar 9, 2017 at 12:31 PM, Ariel Weisberg  wrote:

> Hi,
>
> Before this change I had already been queuing the jobs myself as a
> reviewer. It also happens to be that many reviewers are committers. I
> wouldn't ask contributors to run the dtests/utests for any purpose other
> then so that they know the submission is done.
>
> Even if they did and they pass it doesn't matter. It only matters if it
> passes in CI. If it fails in CI but passes on their desktop it's not
> good enough so we have to run in CI anyways.
>
> If a reviewer is not a committer. Well they can ask someone else to do
> it? I know we have issues with responsiveness, but I would make myself
> available for that. It shouldn't be a big problem because if someone is
> doing a lot of reviews they should be a committer right?
>
> Regards,
> Ariel
>
> On Thu, Mar 9, 2017, at 01:51 PM, Jason Brown wrote:
> > Hey all,
> >
> > A nice convention we've stumbled into wrt to patches submitted via Jira
> > is
> > to post the results of unit test and dtest runs to the ticket (to show
> > the
> > patch doesn't break things). Many contributors have used the
> > DataStax-provided cassci system, but that's not the best long term
> > solution. To that end, I'd like to start a conversation about what is the
> > best way to proceed going forward, and then add it to the "How to
> > contribute" docs.
> >
> > As an example, should contributors/committers run dtests and unit tests
> > on
> > *some* machine (publicly available or otherwise), and then post those
> > results to the ticket? This could be a link to a build system, like what
> > we
> > have with cassci, or just  upload the output of the test run(s).
> >
> > I don't have any fixed notions, and am looking forward to hearing other's
> > ideas.
> >
> > Thanks,
> >
> > -Jason
> >
> > p.s. a big thank you to DataStax for providing the cassci system
>

jira ticket assignment

[Jason Brown]

Hey all,

Currently, when a new contributor wants to have a ticket assigned to
themselves, they cannot due to JIRA permissions. The contributor (or
someone on behalf of the contributor) usually needs to contact the
dev-channel and ask for the magical permissions to be granted, and there's
only a handful of folks who can grant that.

Is anyone aware of why this restriction is in place, or feel strongly it
should exist? I'd like to change the JIRA defaults and let anyone assign
themselves to a ticket (see
https://issues.apache.org/jira/browse/INFRA-11950 for an example of another
Apache project that has done this already).

Thoughts?

If no strong objections I'll set up a PMC vote within a day or two.

Thanks,

-Jason

Re: Code quality, principles and rules

[Jason Brown]

>> do we have plan to integrate with a dependency injection framework?

No, we (the maintainers) have been pretty much against more frameworks due
to performance reasons, overhead, and dependency management problems.

On Thu, Mar 16, 2017 at 2:04 PM, Qingcun Zhou  wrote:

> Since we're here, do we have plan to integrate with a dependency injection
> framework like Dagger2? Otherwise it'll be difficult to write unit test
> cases.
>
> On Thu, Mar 16, 2017 at 1:16 PM, Edward Capriolo 
> wrote:
>
> > On Thu, Mar 16, 2017 at 3:10 PM, Jeff Jirsa  wrote:
> >
> > >
> > >
> > > On 2017-03-16 10:32 (-0700), François Deliège 
> > > wrote:
> > > >
> > > > To get this started, here is an initial proposal:
> > > >
> > > > Principles:
> > > >
> > > > 1. Tests always pass.  This is the starting point. If we don't care
> > > about test failures, then we should stop writing tests. A recurring
> > failing
> > > test carries no signal and is better deleted.
> > > > 2. The code is tested.
> > > >
> > > > Assuming we can align on these principles, here is a proposal for
> their
> > > implementation.
> > > >
> > > > Rules:
> > > >
> > > > 1. Each new release passes all tests (no flakinesss).
> > > > 2. If a patch has a failing test (test touching the same code path),
> > the
> > > code or test should be fixed prior to being accepted.
> > > > 3. Bugs fixes should have one test that fails prior to the fix and
> > > passes after fix.
> > > > 4. New code should have at least 90% test coverage.
> > > >
> > > First I was
> > > I agree with all of these and hope they become codified and followed. I
> > > don't know anyone who believes we should be committing code that breaks
> > > tests - but we should be more strict with requiring green test runs,
> and
> > > perhaps more strict with reverting patches that break tests (or cause
> > them
> > > to be flakey).
> > >
> > > Ed also noted on the user list [0] that certain sections of the code
> > > itself are difficult to test because of singletons - I agree with the
> > > suggestion that it's time to revisit CASSANDRA-7837 and CASSANDRA-10283
> > >
> > > Finally, we should also recall Jason's previous notes [1] that the
> actual
> > > test infrastructure available is limited - the system provided by
> > Datastax
> > > is not generally open to everyone (and not guaranteed to be permanent),
> > and
> > > the infrastructure currently available to the ASF is somewhat limited
> > (much
> > > slower, at the very least). If we require tests passing (and I agree
> that
> > > we should), we need to define how we're going to be testing (or how
> we're
> > > going to be sharing test results), because the ASF hardware isn't going
> > to
> > > be able to do dozens of dev branch dtest runs per day in its current
> > form.
> > >
> > > 0: https://lists.apache.org/thread.html/f6f3fc6d0ad1bd54a6185ce7bd7a2f
> > > 6f09759a02352ffc05df92eef6@%3Cuser.cassandra.apache.org%3E
> > > 1: https://lists.apache.org/thread.html/5fb8f0446ab97644100e4ef987f36e
> > > 07f44e8dd6d38f5dc81ecb3cdd@%3Cdev.cassandra.apache.org%3E
> > >
> > >
> > >
> > Ed also noted on the user list [0] that certain sections of the code
> itself
> > are difficult to test because of singletons - I agree with the suggestion
> > that it's time to revisit CASSANDRA-7837 and CASSANDRA-10283
> >
> > Thanks for the shout out!
> >
> > I was just looking at a patch about compaction. The patch was to
> calculate
> > free space correctly in case X. Compaction is not something that requires
> > multiple nodes to test. The logic on the surface seems simple: find
> tables
> > of similar size and select them and merge them. The reality is it turns
> out
> > now to be that way. The coverage itself both branch and line may be very
> > high, but what the code does not do is directly account for a wide
> variety
> > of scenarios. Without direct tests you end up with a mental approximation
> > of what it does and that varies from person to person and accounts for
> the
> > cases that fit in your mind. For example, you personally are only running
> > LevelDB inspired compaction.
> >
> > Being that this this is not a multi-node problem you should be able to re
> > factor this heavily. Pulling everything out to a static method where all
> > the parameters are arguments, or inject a lot of mocks in the current
> code,
> > and develop some scenario based coverage.
> >
> > That is how i typically "rescue" code I take over. I look at the
> nightmare
> > and say, "damn i am really afraid to touch this". I construct 8 scenarios
> > that test green. Then I force some testing into it through careful re
> > factoring. Now, I probably know -something- about it. Now, you are fairly
> > free to do a wide ranging refactor, because you at least counted for 8
> > scenarios and you put unit test traps so that some rules are enforced.
> (Or
> > the person changing the code has to actively REMOVE your tests asserting
> it
> > was not or no longer is valid). Later on you (or someone els

Re: Code quality, principles and rules

[Jason Brown]

As someone who spent a lot of time looking at the singletons topic in the
past, Blake brings a great perspective here. Figuring out and communicating
how best to test with the system we have (and of course incrementally
making that system easier to work with/test) seems like an achievable goal.

On Fri, Mar 17, 2017 at 10:17 AM, Edward Capriolo 
wrote:

> On Fri, Mar 17, 2017 at 12:33 PM, Blake Eggleston 
> wrote:
>
> > I think we’re getting a little ahead of ourselves talking about DI
> > frameworks. Before that even becomes something worth talking about, we’d
> > need to have made serious progress on un-spaghettifying Cassandra in the
> > first place. It’s an extremely tall order. Adding a DI framework right
> now
> > would be like throwing gasoline on a raging tire fire.
> >
> > Removing singletons seems to come up every 6-12 months, and usually
> > abandoned once people figure out how difficult they are to remove
> properly.
> > I do think removing them *should* be a long term goal, but we really need
> > something more immediately actionable. Otherwise, nothing’s going to
> > happen, and we’ll be having this discussion again in a year or so when
> > everyone’s angry that Cassandra 5.0 still isn’t ready for production, a
> > year after it’s release.
> >
> > That said, the reason singletons regularly get brought up is because
> doing
> > extensive testing of anything in Cassandra is pretty much impossible,
> since
> > the code is basically this big web of interconnected global state.
> Testing
> > anything in isolation can’t be done, which, for a distributed database,
> is
> > crazy. It’s a chronic problem that handicaps our ability to release a
> > stable database.
> >
> > At this point, I think a more pragmatic approach would be to draft and
> > enforce some coding standards that can be applied in day to day
> development
> > that drive incremental improvement of the testing and testability of the
> > project. What should be tested, how it should be tested. How to write new
> > code that talks to the rest of Cassandra and is testable. How to fix bugs
> > in old code in a way that’s testable. We should also have some guidelines
> > around refactoring the wildly untested sections, how to get started, what
> > to do, what not to do, etc.
> >
> > Thoughts?
>
>
> To make the conversation practical. There is one class I personally really
> want to refactor so it can be tested:
>
> https://github.com/apache/cassandra/blob/trunk/src/java/
> org/apache/cassandra/net/OutboundTcpConnection.java
>
> There is little coverage here. Questions like:
> what errors cause the connection to restart?
> when are undropable messages are dropped?
> what happens when the queue fills up?
> Infamous throw new AssertionError(ex); (which probably bubble up to
> nowhere)
> what does the COALESCED strategy do in case XYZ.
> A nifty label (wow a label you just never see those much!)
> outer:
> while (!isStopped)
>
> Comments to jira's that probably are not explicitly tested:
> // If we haven't retried this message yet, put it back on the queue to
> retry after re-connecting.
> // See CASSANDRA-5393 and CASSANDRA-12192.
>
> If I were to undertake this cleanup, would there actually be support? IE if
> this going to turn into an "it aint broken. don't fix it thing" or a "we
> don't want to change stuff just to add tests" . Like will someone pledge to
> agree its kinda wonky and merge the effort in < 1 years time?
>

Re: Code quality, principles and rules

[Jason Brown]

To François's point about code coverage for new code, I think this makes a
lot of sense wrt large features (like the current work on 8457/12229/9754).
It's much simpler to (mentally, at least) isolate those changed sections
and it'll show up better in a code coverage report. With small patches,
that might be harder to achieve - however, as the patch should come with
*some* tests (unless it's a truly trivial patch), it might just work itself
out.

On Fri, Mar 17, 2017 at 11:19 AM, Jason Brown  wrote:

> As someone who spent a lot of time looking at the singletons topic in the
> past, Blake brings a great perspective here. Figuring out and communicating
> how best to test with the system we have (and of course incrementally
> making that system easier to work with/test) seems like an achievable goal.
>
> On Fri, Mar 17, 2017 at 10:17 AM, Edward Capriolo 
> wrote:
>
>> On Fri, Mar 17, 2017 at 12:33 PM, Blake Eggleston 
>> wrote:
>>
>> > I think we’re getting a little ahead of ourselves talking about DI
>> > frameworks. Before that even becomes something worth talking about, we’d
>> > need to have made serious progress on un-spaghettifying Cassandra in the
>> > first place. It’s an extremely tall order. Adding a DI framework right
>> now
>> > would be like throwing gasoline on a raging tire fire.
>> >
>> > Removing singletons seems to come up every 6-12 months, and usually
>> > abandoned once people figure out how difficult they are to remove
>> properly.
>> > I do think removing them *should* be a long term goal, but we really
>> need
>> > something more immediately actionable. Otherwise, nothing’s going to
>> > happen, and we’ll be having this discussion again in a year or so when
>> > everyone’s angry that Cassandra 5.0 still isn’t ready for production, a
>> > year after it’s release.
>> >
>> > That said, the reason singletons regularly get brought up is because
>> doing
>> > extensive testing of anything in Cassandra is pretty much impossible,
>> since
>> > the code is basically this big web of interconnected global state.
>> Testing
>> > anything in isolation can’t be done, which, for a distributed database,
>> is
>> > crazy. It’s a chronic problem that handicaps our ability to release a
>> > stable database.
>> >
>> > At this point, I think a more pragmatic approach would be to draft and
>> > enforce some coding standards that can be applied in day to day
>> development
>> > that drive incremental improvement of the testing and testability of the
>> > project. What should be tested, how it should be tested. How to write
>> new
>> > code that talks to the rest of Cassandra and is testable. How to fix
>> bugs
>> > in old code in a way that’s testable. We should also have some
>> guidelines
>> > around refactoring the wildly untested sections, how to get started,
>> what
>> > to do, what not to do, etc.
>> >
>> > Thoughts?
>>
>>
>> To make the conversation practical. There is one class I personally really
>> want to refactor so it can be tested:
>>
>> https://github.com/apache/cassandra/blob/trunk/src/java/org/
>> apache/cassandra/net/OutboundTcpConnection.java
>>
>> There is little coverage here. Questions like:
>> what errors cause the connection to restart?
>> when are undropable messages are dropped?
>> what happens when the queue fills up?
>> Infamous throw new AssertionError(ex); (which probably bubble up to
>> nowhere)
>> what does the COALESCED strategy do in case XYZ.
>> A nifty label (wow a label you just never see those much!)
>> outer:
>> while (!isStopped)
>>
>> Comments to jira's that probably are not explicitly tested:
>> // If we haven't retried this message yet, put it back on the queue to
>> retry after re-connecting.
>> // See CASSANDRA-5393 and CASSANDRA-12192.
>>
>> If I were to undertake this cleanup, would there actually be support? IE
>> if
>> this going to turn into an "it aint broken. don't fix it thing" or a "we
>> don't want to change stuff just to add tests" . Like will someone pledge
>> to
>> agree its kinda wonky and merge the effort in < 1 years time?
>>
>
>

Re: [VOTE] Ask Infra to move github notification emails to commits@

[Jason Brown]

+1

On Mon, Mar 20, 2017 at 3:02 PM, Brandon Williams  wrote:

> +1, we've had to explain this a thousand times here.
>
> On Mon, Mar 20, 2017 at 5:00 PM, Jeff Jirsa  wrote:
>
> > There's no reason for the dev list to get spammed everytime there's a
> > github PR. We know most of the time we prefer JIRAs for real code PRs,
> but
> > with docs being in tree and low barrier to entry, we may want to accept
> > docs through PRs ( see https://issues.apache.org/
> > jira/browse/CASSANDRA-13256
> > , and comment on it if you disagree).
> >
> > To make that viable, we should make it not spam dev@ with every comment.
> > Therefore I propose we move github PR comments/actions to commits@ so as
> > not to clutter the dev@ list.
> >
> > Voting to remain open for 72 hours.
> >
> > - Jeff
> >
>

Re: [VOTE] Ask Infra to move github notification emails to pr@

[Jason Brown]

+1
On Mon, Mar 20, 2017 at 18:21 Anthony Grasso 
wrote:

> +1
>
> On 21 March 2017 at 09:32, Jeff Jirsa  wrote:
>
> > There's no reason for the dev list to get spammed everytime there's a
> > github PR. We know most of the time we prefer JIRAs for real code PRs,
> but
> > with docs being in tree and low barrier to entry, we may want to accept
> > docs through PRs ( see https://issues.apache.org/
> > jira/browse/CASSANDRA-13256
> > , and comment on it if you disagree).
> >
> > To make that viable, we should make it not spam dev@ with every comment.
> > Therefore I propose we move github PR comments/actions to pr@ so as
> > not to clutter the dev@ list.
> >
> > Voting to remain open for 72 hours.
> >
> > - Jeff
> >
>

Re: [DISCUSS] Implementing code quality principles, and rules (was: Code quality, principles and rules)

[Jason Brown]

re: code coverage (this could become it's own thread, and probably should)

As somebody working on a new large feature (CASSANDRA-8457), I have a few
thoughts/observations about code coverage. As a preface, though, I'll point
out that we have jacoco (http://www.jacoco.org/jacoco/trunk/index.html)
already incorporated into the build, so it's quite easy to generate code
coverage reports ('ant codecoverage' runs the tests and creates the jacoco
binary, and 'ant jacoco-report' creates an html dump).

As CASSANDRA-8457 is largely additive, it's pretty easy for me to confirm
the code coverage, as it's a whole new package - so clearly it's pretty
obvious to what i didn't cover in the test suite. Contrast this against the
rest of the code base. It may be harder to discern, from a simple
percentage output, just how much of an impact the added tests have across
the entire code base. Further, if a given patch added minor functionality
(say a one-liner fix), plus implementing some toString() functions, you
might actually see the coverage go *down* if the patch doesn't include
tests for the toString().

Also, for things that just can't be unit tested in the current code base,
but where correctness can be ensured with a dtest, that wouldn't get
included by coverage tools. Of course, this is where incremental changes to
the code base to allow unit testing are really awesome and appreciated.
I've been encouraging some folks to do that with their submissions. It
makes their patches a bit more involved (for them to code and for me to
review), but hopefully it improves the overall quality of the code.

Thus, imho, for new/large features, sure, having a high coverage is an
awesome thing: I'd think a minimum of 80%, with a "highly recommended"
target of > 90%. For smaller fixes, I'm not so sure, but I think a
reasonable agreement between the contributor and reviewer, based on the
guidelines coming from Blake's forthcoming doc, seem like a good point of
departure.

On Tue, Mar 28, 2017 at 5:57 AM, Josh McKenzie  wrote:

> Hold on now - there's no reason to throw the baby out with the bathwater on
> this. Code coverage on it's own is admittedly a questionable metric, but
> bad metrics are better than *no* metrics if taken with context.
>
> Most code coverage tools will allow you to drill down to the granularity
> we'd need to confirm that new code (new, as in entirely new file, etc) was
> covered, and to delta against a previously run coverage metric to make sure
> there's an uptick in modules where we make changes. It's quite a bit more
> involved / extra overhead compared to what we do now; I'd argue we probably
> need to learn to crawl before we can walk (as in, actually write tests for
> the code that goes into the project and do a *lot* more jmh benchmarking
> for perf changes).
>
> Definitely like the direction this is headed, but it sounds like some
> cart-before-horse to me if we integrate point 4 with the rest of them. Also
> - for point 2 who's going to do the work to correlate flaky failures to the
> changes that introduced them, or are we just going to accept that flaky
> tests get added and deal with them retroactively? I assume we could also
> set up some kind of CI job that would run a new test, say, 20 times and
> greenlight it for patches adding completely new tests? Might be a bad idea,
> might not.
>
> On Tue, Mar 28, 2017 at 1:13 AM, Blake Eggleston 
> wrote:
>
> > In addition to it’s test coverage problem, the project has a general
> > testability problem, and I think it would be more effective to introduce
> > some testing guidelines and standards that drive incremental improvement
> of
> > both, instead of requiring an arbitrary code coverage metric be hit,
> which
> > doesn’t tell the whole story anyway.
> >
> > It’s not ready yet, but I’ve been putting together a testing standards
> > document for the project since bringing it up in the “Code quality,
> > principles and rules” email thread a week or so ago.
> >
> > On March 27, 2017 at 4:51:31 PM, Edward Capriolo (edlinuxg...@gmail.com)
> > wrote:
> > On Mon, Mar 27, 2017 at 7:03 PM, Josh McKenzie 
> > wrote:
> >
> > > How do we plan on verifying #4? Also, root-cause to tie back new code
> > that
> > > introduces flaky tests (i.e. passes on commit, fails 5% of the time
> > > thereafter) is a non-trivial pursuit (thinking #2 here), and a pretty
> > > common problem in this environment.
> > >
> > > On Mon, Mar 27, 2017 at 6:51 PM, Nate McCall 
> wrote:
> > >
> > > > I don't want to lose track of the original idea from François, so
> > > > let's do this formally in preparation for a vote. Having this all in
> > > > place will make transition to new testing infrastructure more
> > > > goal-oriented and keep us more focused moving forward.
> > > >
> > > > Does anybody have specific feedback/discussion points on the
> following
> > > > (awesome, IMO) proposal:
> > > >
> > > > Principles:
> > > >
> > > > 1. Tests always pass. This is the starting point. If we d

[VOTE] self-assignment of jira tickets

[Jason Brown]

Hey all,

Following up on my emai

Re: [VOTE] self-assignment of jira tickets

[Jason Brown]

oops, hit sent too early. will send another email shortly

On Wed, Mar 29, 2017 at 6:14 AM, Jason Brown  wrote:

> Hey all,
>
> Following up on my emai
>

[VOTE] self-assignment of jira tickets

[Jason Brown]

Hey all,

Following up my thread from a week or two ago (
https://lists.apache.org/thread.html/0665f40c7213654e99817141972c003a2131aba7a1c63d6765db75c5@%3Cdev.cassandra.apache.org%3E),
I'd like to propose a vote to change to allow any potential contributor to
assign a jira to themselves without needing to be added to the contributors
group first.

https://issues.apache.org/jira/browse/INFRA-11950 is an example of how to
get this done with INFRA.

Vote will be open for 72 hours.

Thanks,

-Jason Brown