+1 Even though I suggested clearing blockers, I'm equally happy with a time-boxed event to draw the line in the sand. As long as its something clear to work towards with appropriate commitment from folk.
On Tue, Apr 3, 2018 at 8:10 AM Sylvain Lebresne <slebre...@apache.org> wrote: > For what it's worth (and based on the project experience), I think the > strategy > of "let's agree on a list of tickets everyone would love to get in before > we > freeze 4.0" doesn't work very well (it's largely useless, expect for making > us > feel good about not releasing anything). Those lists always end up being > too > big especially given we have no control on people's ability to contribute > (some stuffs will always lag for a very long time, even when they sound > really cool on paper). > > I'm also a bit sad that we seem to be getting back to our old demons of > trying > to shove as much as we possibly can in the next major as if having a > feature > miss it means it will never happen. The 4.0 changelog is big already and we > haven't made a release with new features in almost a year now, so I > personally > think we should start being a bit more aggressive with it and learn to get > comfortable letting feature slip if they are not ready. > > My concrete proposal would be to declare a feature freeze for 4.0 in 2 > months, > so say June 1th. That leave some time for finishing features that are in > progress, but not too much to get derailed. And let's be strict on that > freeze. > After that, we'll see how quickly we can get stuffs to stabilize but I'd > suggest aiming for an alpha 3-4 weeks after that. > > Of course, we should probably (re-(re-(re-)))start a discussion on release > "strategy" in parallel because it doesn't seem we have one right now, but > that's imo a discussion we should keep separate. > > -- > Sylvain > > > On Mon, Apr 2, 2018 at 4:54 PM DuyHai Doan <doanduy...@gmail.com> wrote: > > > My wish list: > > > > * Add support for arithmetic operators (CASSANDRA-11935) > > * Allow IN restrictions on column families with collections > > (CASSANDRA-12654) > > * Add support for + and - operations on dates (CASSANDRA-11936) > > * Add the currentTimestamp, currentDate, currentTime and currentTimeUUID > > functions (CASSANDRA-13132) > > * Allow selecting Map values and Set elements (CASSANDRA-7396) > > > > Those are mostly useful for timeseries data models and I guess has no > > significant impact on the internals and operations so the risk of > > regression is low > > > > On Mon, Apr 2, 2018 at 4:33 PM, Jeff Jirsa <jji...@gmail.com> wrote: > > > > > 9608 (java9) > > > > > > -- > > > Jeff Jirsa > > > > > > > > > > On Apr 2, 2018, at 3:45 AM, Jason Brown <jasedbr...@gmail.com> > wrote: > > > > > > > > The only additional tickets I'd like to mention are: > > > > > > > > https://issues.apache.org/jira/browse/CASSANDRA-13971 - Automatic > > > > certificate management using Vault > > > > - Stefan's Vault integration work. A sub-ticket, CASSANDRA-14102, > > > addresses > > > > encryption at-rest, subsumes CASSANDRA-9633 (SSTable encryption) - > > which > > > I > > > > doubt I would be able to get to any time this year. It would > definitely > > > be > > > > nice to have a clarified encryption/security story for 4.0. > > > > > > > > https://issues.apache.org/jira/browse/CASSANDRA-11990 - Address rows > > > rather > > > > than partitions in SASI > > > > - a nice update for SASI, but not critical. > > > > > > > > -Jason > > > > > > > >> On Sat, Mar 31, 2018 at 6:53 PM, Ben Bromhead <b...@instaclustr.com> > > > wrote: > > > >> > > > >> Apologies all, I didn't realize I was responding to this discussion > > > only on > > > >> the @user list. One of the perils of responding to a thread that is > on > > > both > > > >> user and dev... > > > >> > > > >> For context, I have included my response to Kurt's previous > discussion > > > on > > > >> this topic as it only ended up on the user list. > > > >> > > > >> *After some further discussions with folks offline, I'd like to > revive > > > this > > > >> discussion. * > > > >> > > > >> *As Kurt mentioned, to keep it simple I if we can simply build > > consensus > > > >> around what is in for 4.0 and what is out. We can then start the > > > process of > > > >> working off a 4.0 branch towards betas and release candidates. Again > > as > > > >> Kurt mentioned, assigning a timeline to it right now is difficult, > but > > > >> having a firm line in the sand around what features/patches are in, > > then > > > >> limiting future 4.0 work to bug fixes will give folks a less > nebulous > > > >> target to work on. * > > > >> > > > >> *The other thing to mention is that once we have a 4.0 branch to > work > > > off, > > > >> we at Instaclustr have a commitment to dogfooding the release > > > candidates on > > > >> our internal staging and internal production workloads before 4.0 > > > becomes > > > >> generally available. I know other folks have similar commitments and > > > simply > > > >> having a 4.0 branch with a clear list of things that are in or out > > will > > > >> allow everyone to start testing and driving towards a quality > > release. * > > > >> > > > >> *The other thing is that there are already a large number of changes > > > ready > > > >> for 4.0, I would suggest not recommending tickets for 4.0 that have > > not > > > yet > > > >> been finished/have outstanding work unless you are the person > working > > > on it > > > >> (or are offering to work on it instead) and can get it ready for > > review > > > in > > > >> a timely fashion. That way we can build a more realistic working > > target. > > > >> For other major breaking changes, there is always 5.0 or 4.1 or > > > whatever we > > > >> end up doing :)* > > > >> > > > >> Thinking further about it, I would suggest a similar process that > was > > > >> applied to releasing 3.0, in order to get to 4.0: > > > >> > > > >> - Clean up ticket labeling. Move tickets unlikely to make it / be > > > worked > > > >> on for 4.0 to something else (e.g. 4.x or whatever). > > > >> - Tickets labeled 4.0 will be the line in the sand, with some > > trigger > > > >> ("done") event where all features not done by a certain event will > > > >> simply > > > >> move into the next release. For the 3.0 branch, this occurred > after > > a > > > >> large review of 8099. For 4.0 it could simply be resolving all > > current > > > >> blockers/major tickets tagged 4.0... doesn't have to be / nor is > it > > > >> something I would strongly advocate. > > > >> - Once we hit this "done" event. Cut a Cassandra-4.0 branch and > > start > > > >> the alpha/beta/rc cycle from that branch, with only bugfixes going > > > into > > > >> it > > > >> - This, in my mind, is similar to the 3.0 approach > > > >> https://mail-archives.apache.org/mod_mbox/cassandra-dev/ > > > >> 201503.mbox/%3CCALdd-zjAyiTbZksMeq2LxGwLF5LPhoi_ > > > >> 4vsjy8JBHBRnsxH%3D8A%40mail.gmail.com%3E, > > > >> but without the subsequent tick-tock :) > > > >> > > > >> There are currently 3 open blockers tagged 4.0, some are old and > > > probably > > > >> not really blockers anymore, there are other tickets that may/should > > be > > > >> blockers on 4.0: > > > >> > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13951 > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13994 > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-12042 > > > >> > > > >> In terms of major tickets that I would like to see land: > > > >> > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-7622 Virtual > > Tables > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13628 Internode > > > netty > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-13475 Pluggable > > > >> Storage > > > >> - https://issues.apache.org/jira/browse/CASSANDRA-9633 SSTable > > > >> encryption > > > >> > > > >> Ben > > > >> > > > >>> On Mon, Feb 12, 2018 at 11:26 PM Jeff Jirsa <jji...@gmail.com> > > wrote: > > > >>> > > > >>> > > > >>> Advantages of cutting a release sooner than later: > > > >>> 1) The project needs to constantly progress forward. Releases are > the > > > >> most > > > >>> visible part of that. > > > >>> 2) Having a huge changelog in a release increases the likelihood of > > > bugs > > > >>> that take time to find. > > > >>> > > > >>> Advantages of a slower release: > > > >>> 1) We don't do major versions often, and when we do breaking > changes > > > >>> (protocol, file format, etc), we should squeeze in as many as > > possible > > > to > > > >>> avoid having to roll new majors > > > >>> 2) There are probably few people actually running 3.11 at scale, so > > > >>> probably few people actually testing trunk. > > > >>> > > > >>> In terms of "big" changes I'd like to see land, the ones that come > to > > > >> mind > > > >>> are: > > > >>> > > > >>> https://issues.apache.org/jira/browse/CASSANDRA-9754 - "Birch" > > > (changes > > > >>> file format) > > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13442 - Transient > > > >>> Replicas (probably adds new replication strategy or similar) > > > >>> https://issues.apache.org/jira/browse/CASSANDRA-13628 - Rest of > the > > > >>> internode netty stuff (no idea if this changes internode stuff, > but I > > > bet > > > >>> it's a lot easier if it lands on a major) > > > >>> https://issues.apache.org/jira/browse/CASSANDRA-7622 - Virtual > > Tables > > > >>> (selfish inclusion, probably doesn't need to be a major at all, > and I > > > >>> wouldn't even lose sleep if it slips, but I'd like to see it land) > > > >>> > > > >>> Stuff I'm ok with slipping to 4.X or 5.0, but probably needs to > land > > > on a > > > >>> major because we'll change something big (like gossip, or the way > > > schema > > > >> is > > > >>> passed, etc): > > > >>> > > > >>> https://issues.apache.org/jira/browse/CASSANDRA-9667 - Strongly > > > >>> consistent membership > > > >>> https://issues.apache.org/jira/browse/CASSANDRA-10699 - Strongly > > > >>> consistent schema > > > >>> > > > >>> All that said, what I really care about is building confidence in > the > > > >>> release, which means an extended testing cycle. If all of those > > patches > > > >>> landed tomorrow, I'd still expect us to be months away from a > > release, > > > >>> because we need to bake the next major - there's too many changes > to > > > >> throw > > > >>> out an alpha/beta/rc and hope someone actually runs it. > > > >>> > > > >>> I don't believe Q3/Q4 is realistic, but I may be biased (or jaded). > > > It's > > > >>> possible Q3/Q4 alpha/beta is realistic, but definitely not a > release. > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> On Sun, Feb 11, 2018 at 8:29 PM, kurt greaves < > k...@instaclustr.com> > > > >>> wrote: > > > >>> > > > >>>> Hi friends, > > > >>>> *TL;DR: Making a plan for 4.0, ideally everyone interested should > > > >> provide > > > >>>> up to two lists, one for tickets they can contribute resources to > > > >> getting > > > >>>> finished, and one for features they think would be desirable for > > 4.0, > > > >> but > > > >>>> not necessarily have the resources to commit to helping with.* > > > >>>> > > > >>>> So we had that Roadmap for 4.0 discussion last year, but there was > > > never > > > >>>> a conclusion or a plan that came from it. Times getting on and the > > > >> changes > > > >>>> list for 4.0 is getting pretty big. I'm thinking it would probably > > > make > > > >>>> sense to define some goals to getting 4.0 released/have an actual > > > plan. > > > >> 4.0 > > > >>>> is already going to be quite an unwieldy release with a lot of > > testing > > > >>>> required. > > > >>>> > > > >>>> Note: the following is open to discussion, if people don't like > the > > > plan > > > >>>> feel free to speak up. But in the end it's a pretty basic plan > and I > > > >> don't > > > >>>> think we should over-complicate it, I also don't want to end up > in a > > > >>>> discussion where we "make a plan to make a plan". Regardless of > > > whatever > > > >>>> plan we do end up following it would still be valuable to have a > > list > > > of > > > >>>> tickets for 4.0 which is the overall goal of this email - so let's > > not > > > >> get > > > >>>> too worked up on the details just yet (save that for after I > > > >>>> summarise/follow up). > > > >>>> > > > >>>> // TODO > > > >>>> I think the best way to go about this would be for us to come up > > with > > > a > > > >>>> list of JIRA's that we want included in 4.0, tag these as 4.0, and > > all > > > >>>> other improvements as 4.x. We can then aim to release 4.0 once all > > the > > > >> 4.0 > > > >>>> tagged tickets (+bug fixes/blockers) are complete. > > > >>>> > > > >>>> Now, the catch is that we obviously don't want to include too many > > > >>>> tickets in 4.0, but at the same time we want to make sure 4.0 has > an > > > >>>> appealing feature set for both users/operators/developers. To > > minimise > > > >>>> scope creep I think the following strategy will help: > > > >>>> > > > >>>> We should maintain two lists: > > > >>>> > > > >>>> 1. JIRA's that people want in 4.0 and can commit resources to > > > getting > > > >>>> them implemented in 4.0. > > > >>>> 2. JIRA's that people simply think would be desirable for 4.0, > but > > > >>>> currently don't have anyone assigned to them or planned > > assignment. > > > >> It > > > >>>> would probably make sense to label these with an additional tag > in > > > >> JIRA. *(User's > > > >>>> please feel free to point out what you want here)* > > > >>>> > > > >>>> From list 1 will come our source of truth for when we release 4.0. > > > >> (after > > > >>>> aggregating a list I will summarise and we can vote on it). > > > >>>> > > > >>>> List 2 would be the "hopeful" list, where stories can be picked up > > > from > > > >>>> if resourcing allows, or where someone comes along and decides > it's > > > good > > > >>>> enough to work on. I guess we can also base this on a vote system > if > > > we > > > >>>> reach the point of including some of them. (but for the moment > it's > > > >> purely > > > >>>> to get an idea of what users actually want). > > > >>>> > > > >>>> Please don't refrain from listing something that's already been > > > >>>> mentioned. The purpose is to get an idea of everyone's > > > >> priorities/interests > > > >>>> and the resources available. We will need multiple resources for > > each > > > >>>> ticket, so anywhere we share an interest will make for a lot > easier > > > work > > > >>>> sharing. > > > >>>> > > > >>>> Note that we are only talking about improvements here. Bugs will > be > > > >>>> treated the same as always, and major issues/regressions we'll > need > > to > > > >> fix > > > >>>> prior to 4.0 anyway. > > > >>>> > > > >>>> TIME FRAME > > > >>>> Generally I think it's a bad idea to commit to any hard deadline, > > but > > > we > > > >>>> should have some time frames in mind. My idea would be to aim for > a > > > Q3/4 > > > >>>> 2018 release, and as we go we just review the outstanding > > improvements > > > >> and > > > >>>> decide whether it's worth pushing it back or if we've got enough > to > > > >>>> release. I suppose keep this time frame in mind when choosing your > > > >> tickets. > > > >>>> > > > >>>> We can aim for an earlier date (midyear?) but I figure the > > > >>>> testing/validation/bugfixing period prior to release might drag > on a > > > >> bit so > > > >>>> being a bit conservative here. > > > >>>> The main goal would be to not let list 1 grow unless we're well > > ahead, > > > >>>> and only cull from it if we're heavily over-committed or we decide > > the > > > >>>> improvement can wait. I assume this all sounds like common sense > but > > > >>>> figured it's better to spell it out now. > > > >>>> > > > >>>> > > > >>>> NEXT STEPS > > > >>>> After 2 weeks/whenever the discussion dies off I'll consolidate > all > > > the > > > >>>> tickets, relevant comments and follow up with a summary, where we > > can > > > >>>> discuss/nitpick issues and come up with a final list to go ahead > > with. > > > >>>> > > > >>>> On a side note, in conjunction with this effort we'll obviously > have > > > to > > > >>>> do something about validation and testing. I'll keep that out of > > this > > > >> email > > > >>>> for now, but there will be a follow up so that those of us willing > > to > > > >> help > > > >>>> validate/test trunk can avoid duplicating effort. > > > >>>> > > > >>>> REVIEW > > > >>>> This is the list of "huge/breaking" tickets that got mentioned in > > the > > > >>>> last roadmap discussion and their statuses. This is not terribly > > > >> important > > > >>>> but just so we can keep in mind what we previously talked about. I > > > >> think we > > > >>>> leave it up to the relevant contributors to decide whether they > want > > > to > > > >> get > > > >>>> the still open tickets into 4.0. > > > >>>> > > > >>>> CASSANDRA-9425 Immutable node-local schema > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9425> - > Committed > > > >>>> CASSANDRA-10699 Strongly consistent schema alterations > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10699> - Open, > no > > > >>>> discussion in quite some time. > > > >>>> CASSANDRA-12229 NIO streaming > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12229> - > Committed > > > >>>> CASSANDRA-8457 NIO messaging > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8457> - > Committed > > > >>>> CASSANDRA-12345 Gossip 2.0 > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12345> - Open, > no > > > sign > > > >>>> of any action. > > > >>>> CASSANDRA-9754 Make index info heap friendly for large CQL > > partitions > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> - In > > progress > > > >> but > > > >>>> no update in a long time. > > > >>>> CASSANDRA-11559 enhanced node representation > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> - Open, > no > > > >>>> change since early 2016. > > > >>>> CASSANDRA-6246 epaxos > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-6246> - In > > progress > > > >> but > > > >>>> no update since Feb 2017. > > > >>>> CASSANDRA-7544 storage port configurable per node > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-7544> - > Committed > > > >>>> CASSANDRA-11115 remove thrift support > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11115> - > Committed > > > >>>> CASSANDRA-10857 dropping compact storage > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10857> - > Committed > > > >>>> > > > >>>> To start us off... > > > >>>> And here are my lists to get us started. > > > >>>> 1. > > > >>>> CASSANDRA-8460 - Tiered/Cold storage for TWCS > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8460> > > > >>>> CASSANDRA-12783 - Batchlog redesign > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12783> > > > >>>> CASSANDRA-11559 - Enchance node representation > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-11559> > > > >>>> CASSANDRA-12344 - Forward writes to replacement node with same > > > >>>> address <https://issues.apache.org/jira/browse/CASSANDRA-12344> > > > >>>> CASSANDRA-8119 - More expressive Consistency Levels > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-8119> > > > >>>> CASSANDRA-14210 - Optimise SSTables upgrade task scheduling > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-14210> > > > >>>> CASSANDRA-10540 - RangeAwareCompaction > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10540> > > > >>>> > > > >>>> > > > >>>> 2: > > > >>>> CASSANDRA-10726 - Read repair inserts should not be blocking > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10726> > > > >>>> CASSANDRA-9754 - Make index info heap friendly for large CQL > > > partitions > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-9754> > > > >>>> CASSANDRA-12294 - LDAP auth > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12294> > > > >>>> CASSANDRA-12151 - Audit logging > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-12151> > > > >>>> CASSANDRA-10495 - Fix streaming with vnodes > > > >>>> <https://issues.apache.org/jira/browse/CASSANDRA-10495> > > > >>>> > > > >>>> Also, here's some handy JQL to start you off: > > > >>>> project = CASSANDRA AND fixVersion in (4.x, 4.0) AND issue in > > > >>>> watchedIssues() AND status != Resolved > > > >>>> > > > >>>> > > > >>> -- > > > >> Ben Bromhead > > > >> CTO | Instaclustr <https://www.instaclustr.com/> > > > >> +1 650 284 9692 <(650)%20284-9692> > > > >> Reliability at Scale > > > >> Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer > > > >> > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > > > > > > -- Ben Bromhead CTO | Instaclustr <https://www.instaclustr.com/> +1 650 284 9692 Reliability at Scale Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
