Apologies all, I didn't realize I was responding to this discussion only on
the @user list. One of the perils of responding to a thread that is on both
user and dev...
For context, I have included my response to Kurt's previous discussion on
this topic as it only ended up on the user list.
*After some further discussions with folks offline, I'd like to revive this
discussion. *
*As Kurt mentioned, to keep it simple I if we can simply build consensus
around what is in for 4.0 and what is out. We can then start the process of
working off a 4.0 branch towards betas and release candidates. Again as
Kurt mentioned, assigning a timeline to it right now is difficult, but
having a firm line in the sand around what features/patches are in, then
limiting future 4.0 work to bug fixes will give folks a less nebulous
target to work on. *
*The other thing to mention is that once we have a 4.0 branch to work off,
we at Instaclustr have a commitment to dogfooding the release candidates on
our internal staging and internal production workloads before 4.0 becomes
generally available. I know other folks have similar commitments and simply
having a 4.0 branch with a clear list of things that are in or out will
allow everyone to start testing and driving towards a quality release. *
*The other thing is that there are already a large number of changes ready
for 4.0, I would suggest not recommending tickets for 4.0 that have not yet
been finished/have outstanding work unless you are the person working on it
(or are offering to work on it instead) and can get it ready for review in
a timely fashion. That way we can build a more realistic working target.
For other major breaking changes, there is always 5.0 or 4.1 or whatever we
end up doing :)*
Thinking further about it, I would suggest a similar process that was
applied to releasing 3.0, in order to get to 4.0:
- Clean up ticket labeling. Move tickets unlikely to make it / be worked
on for 4.0 to something else (e.g. 4.x or whatever).
- Tickets labeled 4.0 will be the line in the sand, with some trigger
("done") event where all features not done by a certain event will simply
move into the next release. For the 3.0 branch, this occurred after a
large review of 8099. For 4.0 it could simply be resolving all current
blockers/major tickets tagged 4.0... doesn't have to be / nor is it
something I would strongly advocate.
- Once we hit this "done" event. Cut a Cassandra-4.0 branch and start
the alpha/beta/rc cycle from that branch, with only bugfixes going into
it
- This, in my mind, is similar to the 3.0 approach
https://mail-archives.apache.org/mod_mbox/cassandra-dev/201503.mbox/%3CCALdd-zjAyiTbZksMeq2LxGwLF5LPhoi_4vsjy8JBHBRnsxH%3D8A%40mail.gmail.com%3E,
but without the subsequent tick-tock :)
There are currently 3 open blockers tagged 4.0, some are old and probably
not really blockers anymore, there are other tickets that may/should be
blockers on 4.0:
- https://issues.apache.org/jira/browse/CASSANDRA-13951
- https://issues.apache.org/jira/browse/CASSANDRA-13994
- https://issues.apache.org/jira/browse/CASSANDRA-12042
In terms of major tickets that I would like to see land:
- https://issues.apache.org/jira/browse/CASSANDRA-7622 Virtual Tables
- https://issues.apache.org/jira/browse/CASSANDRA-13628 Internode netty
- https://issues.apache.org/jira/browse/CASSANDRA-13475 Pluggable Storage
- https://issues.apache.org/jira/browse/CASSANDRA-9633 SSTable encryption
Ben
On Mon, Feb 12, 2018 at 11:26 PM Jeff Jirsa <jji...@gmail.com> wrote:
>
> Advantages of cutting a release sooner than later:
> 1) The project needs to constantly progress forward. Releases are the most
> visible part of that.
> 2) Having a huge changelog in a release increases the likelihood of bugs
> that take time to find.
>
> Advantages of a slower release:
> 1) We don't do major versions often, and when we do breaking changes
> (protocol, file format, etc), we should squeeze in as many as possible to
> avoid having to roll new majors
> 2) There are probably few people actually running 3.11 at scale, so
> probably few people actually testing trunk.
>
> In terms of "big" changes I'd like to see land, the ones that come to mind
> are:
>
> https://issues.apache.org/jira/browse/CASSANDRA-9754 - "Birch" (changes
> file format)
> https://issues.apache.org/jira/browse/CASSANDRA-13442 - Transient
> Replicas (probably adds new replication strategy or similar)
> https://issues.apache.org/jira/browse/CASSANDRA-13628 - Rest of the
> internode netty stuff (no idea if this changes internode stuff, but I bet
> it's a lot easier if it lands on a major)
> https://issues.apache.org/jira/browse/CASSANDRA-7622 - Virtual Tables
> (selfish inclusion, probably doesn't need to be a major at all, and I
> wouldn't even lose sleep if it slips, but I'd like to see it land)
>
> Stuff I'm ok with slipping to 4.X or 5.0, but probably needs to land on a
> major because we'll change something big (like gossip, or the way schema is
> passed, etc):
>
> https://issues.apache.org/jira/browse/CASSANDRA-9667 - Strongly
> consistent membership
> https://issues.apache.org/jira/browse/CASSANDRA-10699 - Strongly
> consistent schema
>
> All that said, what I really care about is building confidence in the
> release, which means an extended testing cycle. If all of those patches
> landed tomorrow, I'd still expect us to be months away from a release,
> because we need to bake the next major - there's too many changes to throw
> out an alpha/beta/rc and hope someone actually runs it.
>
> I don't believe Q3/Q4 is realistic, but I may be biased (or jaded). It's
> possible Q3/Q4 alpha/beta is realistic, but definitely not a release.
>
>
>
>
> On Sun, Feb 11, 2018 at 8:29 PM, kurt greaves <k...@instaclustr.com>
> wrote:
>
>> Hi friends,
>> *TL;DR: Making a plan for 4.0, ideally everyone interested should provide
>> up to two lists, one for tickets they can contribute resources to getting
>> finished, and one for features they think would be desirable for 4.0, but
>> not necessarily have the resources to commit to helping with.*
>>
>> So we had that Roadmap for 4.0 discussion last year, but there was never
>> a conclusion or a plan that came from it. Times getting on and the changes
>> list for 4.0 is getting pretty big. I'm thinking it would probably make
>> sense to define some goals to getting 4.0 released/have an actual plan. 4.0
>> is already going to be quite an unwieldy release with a lot of testing
>> required.
>>
>> Note: the following is open to discussion, if people don't like the plan
>> feel free to speak up. But in the end it's a pretty basic plan and I don't
>> think we should over-complicate it, I also don't want to end up in a
>> discussion where we "make a plan to make a plan". Regardless of whatever
>> plan we do end up following it would still be valuable to have a list of
>> tickets for 4.0 which is the overall goal of this email - so let's not get
>> too worked up on the details just yet (save that for after I
>> summarise/follow up).
>>
>> // TODO
>> I think the best way to go about this would be for us to come up with a
>> list of JIRA's that we want included in 4.0, tag these as 4.0, and all
>> other improvements as 4.x. We can then aim to release 4.0 once all the 4.0
>> tagged tickets (+bug fixes/blockers) are complete.
>>
>> Now, the catch is that we obviously don't want to include too many
>> tickets in 4.0, but at the same time we want to make sure 4.0 has an
>> appealing feature set for both users/operators/developers. To minimise
>> scope creep I think the following strategy will help:
>>
>> We should maintain two lists:
>>
>> 1. JIRA's that people want in 4.0 and can commit resources to getting
>> them implemented in 4.0.
>> 2. JIRA's that people simply think would be desirable for 4.0, but
>> currently don't have anyone assigned to them or planned assignment. It
>> would probably make sense to label these with an additional tag in JIRA.
>> *(User's
>> please feel free to point out what you want here)*
>>
>> From list 1 will come our source of truth for when we release 4.0. (after
>> aggregating a list I will summarise and we can vote on it).
>>
>> List 2 would be the "hopeful" list, where stories can be picked up from
>> if resourcing allows, or where someone comes along and decides it's good
>> enough to work on. I guess we can also base this on a vote system if we
>> reach the point of including some of them. (but for the moment it's purely
>> to get an idea of what users actually want).
>>
>> Please don't refrain from listing something that's already been
>> mentioned. The purpose is to get an idea of everyone's priorities/interests
>> and the resources available. We will need multiple resources for each
>> ticket, so anywhere we share an interest will make for a lot easier work
>> sharing.
>>
>> Note that we are only talking about improvements here. Bugs will be
>> treated the same as always, and major issues/regressions we'll need to fix
>> prior to 4.0 anyway.
>>
>> TIME FRAME
>> Generally I think it's a bad idea to commit to any hard deadline, but we
>> should have some time frames in mind. My idea would be to aim for a Q3/4
>> 2018 release, and as we go we just review the outstanding improvements and
>> decide whether it's worth pushing it back or if we've got enough to
>> release. I suppose keep this time frame in mind when choosing your tickets.
>>
>> We can aim for an earlier date (midyear?) but I figure the
>> testing/validation/bugfixing period prior to release might drag on a bit so
>> being a bit conservative here.
>> The main goal would be to not let list 1 grow unless we're well ahead,
>> and only cull from it if we're heavily over-committed or we decide the
>> improvement can wait. I assume this all sounds like common sense but
>> figured it's better to spell it out now.
>>
>>
>> NEXT STEPS
>> After 2 weeks/whenever the discussion dies off I'll consolidate all the
>> tickets, relevant comments and follow up with a summary, where we can
>> discuss/nitpick issues and come up with a final list to go ahead with.
>>
>> On a side note, in conjunction with this effort we'll obviously have to
>> do something about validation and testing. I'll keep that out of this email
>> for now, but there will be a follow up so that those of us willing to help
>> validate/test trunk can avoid duplicating effort.
>>
>> REVIEW
>> This is the list of "huge/breaking" tickets that got mentioned in the
>> last roadmap discussion and their statuses. This is not terribly important
>> but just so we can keep in mind what we previously talked about. I think we
>> leave it up to the relevant contributors to decide whether they want to get
>> the still open tickets into 4.0.
>>
>> CASSANDRA-9425 Immutable node-local schema
>> <https://issues.apache.org/jira/browse/CASSANDRA-9425> - Committed
>> CASSANDRA-10699 Strongly consistent schema alterations
>> <https://issues.apache.org/jira/browse/CASSANDRA-10699> - Open, no
>> discussion in quite some time.
>> CASSANDRA-12229 NIO streaming
>> <https://issues.apache.org/jira/browse/CASSANDRA-12229> - Committed
>> CASSANDRA-8457 NIO messaging
>> <https://issues.apache.org/jira/browse/CASSANDRA-8457> - Committed
>> CASSANDRA-12345 Gossip 2.0
>> <https://issues.apache.org/jira/browse/CASSANDRA-12345> - Open, no sign
>> of any action.
>> CASSANDRA-9754 Make index info heap friendly for large CQL partitions
>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> - In progress but
>> no update in a long time.
>> CASSANDRA-11559 enhanced node representation
>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> - Open, no
>> change since early 2016.
>> CASSANDRA-6246 epaxos
>> <https://issues.apache.org/jira/browse/CASSANDRA-6246> - In progress but
>> no update since Feb 2017.
>> CASSANDRA-7544 storage port configurable per node
>> <https://issues.apache.org/jira/browse/CASSANDRA-7544> - Committed
>> CASSANDRA-11115 remove thrift support
>> <https://issues.apache.org/jira/browse/CASSANDRA-11115> - Committed
>> CASSANDRA-10857 dropping compact storage
>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> - Committed
>>
>> To start us off...
>> And here are my lists to get us started.
>> 1.
>> CASSANDRA-8460 - Tiered/Cold storage for TWCS
>> <https://issues.apache.org/jira/browse/CASSANDRA-8460>
>> CASSANDRA-12783 - Batchlog redesign
>> <https://issues.apache.org/jira/browse/CASSANDRA-12783>
>> CASSANDRA-11559 - Enchance node representation
>> <https://issues.apache.org/jira/browse/CASSANDRA-11559>
>> CASSANDRA-12344 - Forward writes to replacement node with same
>> address <https://issues.apache.org/jira/browse/CASSANDRA-12344>
>> CASSANDRA-8119 - More expressive Consistency Levels
>> <https://issues.apache.org/jira/browse/CASSANDRA-8119>
>> CASSANDRA-14210 - Optimise SSTables upgrade task scheduling
>> <https://issues.apache.org/jira/browse/CASSANDRA-14210>
>> CASSANDRA-10540 - RangeAwareCompaction
>> <https://issues.apache.org/jira/browse/CASSANDRA-10540>
>>
>>
>> 2:
>> CASSANDRA-10726 - Read repair inserts should not be blocking
>> <https://issues.apache.org/jira/browse/CASSANDRA-10726>
>> CASSANDRA-9754 - Make index info heap friendly for large CQL partitions
>> <https://issues.apache.org/jira/browse/CASSANDRA-9754>
>> CASSANDRA-12294 - LDAP auth
>> <https://issues.apache.org/jira/browse/CASSANDRA-12294>
>> CASSANDRA-12151 - Audit logging
>> <https://issues.apache.org/jira/browse/CASSANDRA-12151>
>> CASSANDRA-10495 - Fix streaming with vnodes
>> <https://issues.apache.org/jira/browse/CASSANDRA-10495>
>>
>> Also, here's some handy JQL to start you off:
>> project = CASSANDRA AND fixVersion in (4.x, 4.0) AND issue in
>> watchedIssues() AND status != Resolved
>>
>>
> --
Ben Bromhead
CTO | Instaclustr <https://www.instaclustr.com/>
+1 650 284 9692
Reliability at Scale
Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
