Re: [DISCUSS] The future of CREATE INDEX

[Miklosovic, Stefan]

I don't want to hijack this thread, I just want to say that the point 4) seems 
to be recurring. I second Caleb in saying that transactional metadata would 
probably fix this. Because of the problem of not being sure that all config is 
same, cluster-wide, I basically dropped the effort on CEP-24 because different 
local configurations might compromise the security.


From: Henrik Ingo 
Sent: Wednesday, May 17, 2023 22:32
To: dev@cassandra.apache.org
Subject: Re: [DISCUSS] The future of CREATE INDEX

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.



I have read the thread but chose to reply to the top message...

I'm coming to this with the background of having worked with MySQL, where both 
the storage engine and index implementation had many options, and often of 
course some index types were only available in some engines.

I would humbly suggest:

1. What's up with naming anything "legacy". Calling the current index type "2i" 
seems perfectly fine with me. From what I've heard it can work great for many 
users?

2. It should be possible to always specify the index type explicitly. In other 
words, it should be possible to CREATE CUSTOM INDEX ... USING "2i" (if it isn't 
already)

2b) It should be possible to just say "SAI" or "SASIIndex", not the full Java 
path.

3. It's a fair point that the "CUSTOM" word may make this sound a bit too 
special... The simplest change IMO is to just make the CUSTOM work optional.

4. Benedict's point that a YAML option is per node is a good one... For 
example, you wouldn't want some nodes to create a 2i index and other nodes a 
SAI index for the same index That said, how many other YAML options can you 
think of that would create total chaos if different nodes actually had 
different values for them? For example what if a guardrail allowed some action 
on some nodes but not others?  Maybe what we need is a jira ticket to enforce 
that certain sections of the config must not differ?

5. That said, the default index type could also be a property of the keyspace

6. MySQL allows the DBA to determine the default engine. This seems to work 
well. If the user doesn't care, they don't care, if they do, they use the 
explicit syntax.

henrik


On Wed, May 10, 2023 at 12:45 AM Caleb Rackliffe 
mailto:calebrackli...@gmail.com>> wrote:
Earlier today, Mick started a thread on the future of our index creation DDL on 
Slack:

https://the-asf.slack.com/archives/C018YGVCHMZ/p1683527794220019

At the moment, there are two ways to create a secondary index.

1.) CREATE INDEX [IF NOT EXISTS] [name] ON  ()

This creates an optionally named legacy 2i on the provided table and column.

ex. CREATE INDEX my_index ON kd.tbl(my_text_col)

2.) CREATE CUSTOM INDEX [IF NOT EXISTS] [name] ON  () USING 
 [WITH OPTIONS = ]

This creates a secondary index on the provided table and column using the 
specified 2i implementation class and (optional) parameters.

ex. CREATE CUSTOM INDEX my_index ON ks.tbl(my_text_col) USING 
'StorageAttachedIndex'

(Note that the work on SAI added aliasing, so `StorageAttachedIndex` is 
shorthand for the fully-qualified class name, which is also valid.)

So what is there to discuss?

The concern Mick raised is...

"...just folk continuing to use CREATE INDEX  because they think CREATE CUSTOM 
INDEX is advanced (or just don't know of it), and we leave users doing 2i (when 
they think they are, and/or we definitely want them to be, using SAI)"

To paraphrase, we want people to use SAI once it's available where possible, 
and the default behavior of CREATE INDEX could be at odds w/ that.

The proposal we seem to have landed on is something like the following:

For 5.0:

1.) Disable by default the creation of new legacy 2i via CREATE INDEX.
2.) Leave CREATE CUSTOM INDEX...USING... available by default.

(Note: How this would interact w/ the existing secondary_indexes_enabled YAML 
options isn't clear yet.)

Post-5.0:

1.) Deprecate and eventually remove SASI when SAI hits full feature parity w/ 
it.
2.) Replace both CREATE INDEX and CREATE CUSTOM INDEX w/ something of a hybrid 
between the two. For example, CREATE INDEX...USING...WITH. This would both be 
flexible enough to accommodate index implementation selection and prescriptive 
enough to force the user to make a decision (and wouldn't change the legacy 
behavior of the existing CREATE INDEX). In this world, creating a legacy 2i 
might look something like CREATE INDEX...USING `legacy`.
3.) Eventually deprecate CREATE CUSTOM INDEX...USING.

Eventually we would have a single enabled DDL statement for index creation that 
would be minimal but also explicit/able to handl

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

[Maxim Muzafarov]

Hello everyone,


Thanks for following this thread and the review, all the system
properties have been moved to CassandraRelevantProperties.
So you can find out what it looks like from the following link:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java#L38


I would like to show you a few more steps in this thread so that the
solution is generally complete. As you may have noticed, we have three
types of system properties: cassandra properties used in production
environments, cassandra properties used for testing only, and
non-cassandra properties. I would like to reuse the @Replaces
annotation to rename cassandra-related properties according to the
following pattern: the 'cassandra.' prefix is for production
properties, and the 'cassandra.test' prefix is for testing properties.

This makes the results of the SystemPropertiesTable virtual table look
more natural to users. I thinks we should include this change for the
5.0 release.
WDYT?


The other code clarity minor improvements to do:

1.
Use WithProperties to ensure that system properties are handled
https://issues.apache.org/jira/browse/CASSANDRA-18453

2.
As a draft agreement, the CassandraRelevantProperties and
CassandraRelevantEnv (and probably DatabaseDescriptor) could share the
same interface to access the system properties, configuration
properties, and/or environment variables. The idea is still in draft
form, so I'm mentioning it here to keep it in context. Will come back
to it when more details are available.
This is what it might look like:
https://github.com/apache/cassandra/pull/2300/files#diff-6b7db8438314143a1b6b1c8c58901a4e3954af8cdd294ca8853a1001c1f4R70

On Fri, 31 Mar 2023 at 07:08, Jacek Lewandowski
 wrote:
>
> I'll do
>
> - - -- --- -  -
> Jacek Lewandowski
>
>
> czw., 30 mar 2023 o 22:09 Miklosovic, Stefan  
> napisał(a):
>>
>> Hi list,
>>
>> we are looking for one more committer to take a look at this patch (1, 2).
>>
>> It looks like there is a lot to go through because of number of files 
>> modified (around 200) but changes are really just about moving everything to 
>> CassandraRelevantProperties. I do not think that it should take more than 1 
>> hour of dedicated effort and we are done!
>>
>> Thanks in advance to whoever reviews this.
>>
>> I want to especially thank Maxim for his perseverance in this matter and I 
>> hope we will eventually deliver this work to trunk.
>>
>> (1) https://github.com/apache/cassandra/pull/2046
>> (2) https://issues.apache.org/jira/browse/CASSANDRA-17797
>>
>> Regards
>>
>> Regards
>>
>> 
>> From: Miklosovic, Stefan 
>> Sent: Wednesday, March 22, 2023 14:34
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSS] Moving system property names to the 
>> CassandraRelevantProperties
>>
>> NetApp Security WARNING: This is an external email. Do not click links or 
>> open attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>>
>>
>> Hi Maxim,
>>
>> thanks for letting us know.
>>
>> I reviewed it couple months ago but I can revisit it to double check. We 
>> need the second reviewer. Until we find somebody, we can not merge this.
>>
>> If anybody wants to take a look, it would be awesome. It seems like a lot of 
>> changes / files touched but it is just about centralizing all properties 
>> scattered around the code base into one place.
>>
>> Regards
>>
>> 
>> From: Maxim Muzafarov 
>> Sent: Tuesday, March 21, 2023 22:59
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSS] Moving system property names to the 
>> CassandraRelevantProperties
>>
>> NetApp Security WARNING: This is an external email. Do not click links or 
>> open attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>>
>>
>> Hello everyone,
>>
>> This a friendly reminder that some help is still needed with the review :-)
>> I have resolved all the conflicts that have arisen in the last month or two.
>>
>> If you'd like to invest some time in code clarity, please take a look:
>> https://github.com/apache/cassandra/pull/2046/files
>>
>> On Wed, 8 Feb 2023 at 19:48, Maxim Muzafarov  wrote:
>> >
>> > Hello everyone,
>> >
>> >
>> > We are trying to clean up the source code around the direct use of
>> > system properties and make this use more manageable and transparent.
>> > To achieve this, I have prepared a patch that moves all system
>> > property names to the CassandraRelevantProperties, which in turn makes
>> > some of the properties visible to a user through the
>> > SystemPropertiesTable virtual table.
>> >
>> > The patch has passed a few rounds of review, but we still need another
>> > pair of eyes to make sure we are not missing anything valuable.
>> > Please, take a look at the patch.
>> >
>> > You can find all the changes here:
>> > https://issues.apache.org/jira/browse/CASSANDRA-17797
>> >
>>

Re: [DISCUSS] Feature branch version hygiene

[Josh McKenzie]

CEP-N seems like a good compromise. NextMajorRelease bumps into our 
interchangeable use of "Major" and "Minor" from a semver perspective and could 
get confusing. Suppose we could do NextFeatureRelease, but at that point why 
not just have it linked to the CEP and have the epic set.

On Thu, May 18, 2023, at 12:26 AM, Caleb Rackliffe wrote:
> ...otherwise I'm fine w/ just the CEP name, like "CEP-7" for SAI, etc.
> 
> On Wed, May 17, 2023 at 11:24 PM Caleb Rackliffe  
> wrote:
>> So when a CEP slips, do we have to create a 5.1-cep-N? Could we just have a 
>> version that's "NextMajorRelease" or something like that? It should still be 
>> pretty easy to bulk replace if we have something else to filter on, like 
>> belonging to an epic?
>> 
>> On Wed, May 17, 2023 at 6:42 PM Mick Semb Wever  wrote:
>>> 
>>> 
>>> On Tue, 16 May 2023 at 13:02, J. D. Jordan  
>>> wrote:
 Process question/discussion. Should tickets that are merged to CEP feature 
 branches, like  https://issues.apache.org/jira/browse/CASSANDRA-18204, 
 have a fixver of 5.0 on them After merging to the feature branch?
 
 
 
 For the SAI CEP which is also using the feature branch method the 
 "reviewed and merged to feature branch" tickets seem to be given a version 
 of NA.
 
 
 
 Not sure that's the best “waiting for cep to merge” version either?  But 
 it seems better than putting 5.0 on them to me.
 
 
 
 Why I’m not keen on 5.0 is because if we cut the release today those 
 tickets would not be there.
 
 
 
 What do other people think?  Is there a better version designation we can 
 use?
 
 
 
 On a different project I have in the past made a “version number” in JIRA 
 for each long running feature branch. Tickets merged to the feature branch 
 got the epic ticket number as their version, and then it got updated to 
 the “real” version when the feature branch was merged to trunk.
 
>>> 
>>> 
>>> Thanks for raising the thread, I remember there was some confusion early 
>>> wrt features branches too.
>>> 
>>> To rehash, for everything currently resolved in trunk 5.0 is the correct 
>>> fixVersion.  (And there should be no unresolved issues today with 5.0 
>>> fixVersion, they should be 5.x)
>>> 
>>> 
>>> When alpha1 is cut, then the 5.0-alpha1 fixVersion is created and 
>>> everything with 5.0 also gets  5.0-alpha1. At the same time 5.0-alpha2, 
>>> 5.0-beta, 5.0-rc, 5.0.0 fixVersions are created. Here both 5.0-beta and 
>>> 5.0-rc are blocking placeholder fixVersions: no resolved issues are left 
>>> with this fixVersion the same as the .x placeholder fixVersions. The 5.0.0 
>>> is also used as a blocking version, though it is also an eventual 
>>> fixVersion for resolved tickets. Also note, all tickets up to and including 
>>> 5.0.0 will also have the 5.0 fixVersion.
>>> 
>>> 
>>> 
>>> A particular reason for doing things the way they are is to make it easy 
>>> for the release manager to bulk correct fixVersions, at release time or 
>>> even later, i.e. without having to read the ticket or go talk to authors or 
>>> painstakingly crawl CHANGES.txt.
>>> 
>>> 
>>> 
>>> For feature branches my suggestion is that we create a fixVersion for each 
>>> of them, e.g. 5.0-cep-15
>>> 
>>> Yup, that's your suggestion Jeremiah (I wrote this up on the plane before I 
>>> got to read your post properly).
>>> 
>>> 
>>> 
>>> (As you say) This then makes it easy to see where the code is (or what the 
>>> patch is currently being based on). And when the feature branch is merged 
>>> then it is easy to bulk replace it with trunk's fixVersion, e.g.  
>>> 5.0-cep-15 with 5.0
>>> 
>>> 
>>> 
>>> The NA fixVersion was introduced for the other repositories, e.g. website 
>>> updates.
>>> 

Re: [DISCUSS] Feature branch version hygiene

[Benedict]

I don’t think we should over complicate this with special CEP release targets. If we do, they shouldn’t be versioned.My personal view is that 5.0 should not be used for any resolved tickets - they should go to 5.0-alpha1, since this is the correct release for them. 5.0 can then be the target version, which makes more sense given it isn’t a concrete release.But, in lieu of that, every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty clear what this means. Some tickets that don’t hit 5.0.0 can then be postponed to a later version, but it’s not like this is burdensome. Anything marked feature/improvement and 5.0.x gets bumped to 5.1.x.On 18 May 2023, at 13:58, Josh McKenzie  wrote:CEP-N seems like a good compromise. NextMajorRelease bumps into our interchangeable use of "Major" and "Minor" from a semver perspective and could get confusing. Suppose we could do NextFeatureRelease, but at that point why not just have it linked to the CEP and have the epic set.On Thu, May 18, 2023, at 12:26 AM, Caleb Rackliffe wrote:...otherwise I'm fine w/ just the CEP name, like "CEP-7" for SAI, etc.On Wed, May 17, 2023 at 11:24 PM Caleb Rackliffe  wrote:So when a CEP slips, do we have to create a 5.1-cep-N? Could we just have a version that's "NextMajorRelease" or something like that? It should still be pretty easy to bulk replace if we have something else to filter on, like belonging to an epic?On Wed, May 17, 2023 at 6:42 PM Mick Semb Wever  wrote:On Tue, 16 May 2023 at 13:02, J. D. Jordan  wrote:Process question/discussion. Should tickets that are merged to CEP feature branches, like  https://issues.apache.org/jira/browse/CASSANDRA-18204, have a fixver of 5.0 on them After merging to the feature branch?For the SAI CEP which is also using the feature branch method the "reviewed and merged to feature branch" tickets seem to be given a version of NA.Not sure that's the best “waiting for cep to merge” version either?  But it seems better than putting 5.0 on them to me.Why I’m not keen on 5.0 is because if we cut the release today those tickets would not be there.What do other people think?  Is there a better version designation we can use?On a different project I have in the past made a “version number” in JIRA for each long running feature branch. Tickets merged to the feature branch got the epic ticket number as their version, and then it got updated to the “real” version when the feature branch was merged to trunk.Thanks for raising the thread, I remember there was some confusion early wrt features branches too.To rehash, for everything currently resolved in trunk 5.0 is the correct fixVersion.  (And there should be no unresolved issues today with 5.0 fixVersion, they should be 5.x)When alpha1 is cut, then the 5.0-alpha1 fixVersion is created and everything with 5.0 also gets  5.0-alpha1.  At the same time 5.0-alpha2, 5.0-beta, 5.0-rc, 5.0.0 fixVersions are created.  Here both 5.0-beta and 5.0-rc are blocking placeholder fixVersions: no resolved issues are left with this fixVersion the same as the .x placeholder fixVersions. The 5.0.0 is also used as a blocking version, though it is also an eventual fixVersion for resolved tickets.  Also note, all tickets up to and including 5.0.0 will also have the 5.0 fixVersion.A particular reason for doing things the way they are is to make it easy for the release manager to bulk correct fixVersions, at release time or even later, i.e. without having to read the ticket or go talk to authors or painstakingly crawl CHANGES.txt.For feature branches my suggestion is that we create a fixVersion for each of them, e.g. 5.0-cep-15Yup, that's your suggestion Jeremiah (I wrote this up on the plane before I got to read your post properly).(As you say) This then makes it easy to see where the code is (or what the patch is currently being based on). And when the feature branch is merged then it is easy to bulk replace it with trunk's fixVersion, e.g.  5.0-cep-15 with 5.0The NA fixVersion was introduced for the other repositories, e.g. website updates.

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

[Miklosovic, Stefan]

Hi Maxim,

thanks for bringing this up. I am glad you did the heavy-lifting in / around 
CassandraRelevantProperties and we can build on top of this.

I am fine with @Replaces for Cassandra system properties. After we put 
everything into CassandraRelevantProperties, one can easily see that there are 
great inconsistencies in properties' naming. As we still need to support the 
old names too, using @Replaces, the similar mechanism we used in 
DatabaseDescriptor, seems like the ideal solution.

By the way, when somebody queries system_views.system_properties, it looks very 
strange in CQL shell, the formatting is just broken. EXPAND ON; does not help 
either. It is quite hard to parse this visually if a user wants to see them 
all. The reason is that there is "java.class.path" property for which the value 
is so long that it basically breaks the output.

Another solution would be to fix the output but I am not sure how it would look 
like.

As we are going to rename them to have same prefixes, could not we remodel that 
table as well? For example:

https://gist.github.com/smiklosovic/de662b7faa25e1fdd56805cdb5ba80a7

Feel free to come up with a different approach.

By doing this, it would be way easier to get just Cassandra properties or just 
properties for tests or just Java properties and selecting just the first two 
groups would not break CQLSH. It is nice that it would have same prefix but I 
am trying to find a way how to utilize the same prefix in CQLSH as well.


From: Maxim Muzafarov 
Sent: Thursday, May 18, 2023 12:54
To: dev@cassandra.apache.org
Subject: Re: [DISCUSS] Moving system property names to the 
CassandraRelevantProperties

NetApp Security WARNING: This is an external email. Do not click links or open 
attachments unless you recognize the sender and know the content is safe.




Hello everyone,


Thanks for following this thread and the review, all the system
properties have been moved to CassandraRelevantProperties.
So you can find out what it looks like from the following link:
https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java#L38


I would like to show you a few more steps in this thread so that the
solution is generally complete. As you may have noticed, we have three
types of system properties: cassandra properties used in production
environments, cassandra properties used for testing only, and
non-cassandra properties. I would like to reuse the @Replaces
annotation to rename cassandra-related properties according to the
following pattern: the 'cassandra.' prefix is for production
properties, and the 'cassandra.test' prefix is for testing properties.

This makes the results of the SystemPropertiesTable virtual table look
more natural to users. I thinks we should include this change for the
5.0 release.
WDYT?


The other code clarity minor improvements to do:

1.
Use WithProperties to ensure that system properties are handled
https://issues.apache.org/jira/browse/CASSANDRA-18453

2.
As a draft agreement, the CassandraRelevantProperties and
CassandraRelevantEnv (and probably DatabaseDescriptor) could share the
same interface to access the system properties, configuration
properties, and/or environment variables. The idea is still in draft
form, so I'm mentioning it here to keep it in context. Will come back
to it when more details are available.
This is what it might look like:
https://github.com/apache/cassandra/pull/2300/files#diff-6b7db8438314143a1b6b1c8c58901a4e3954af8cdd294ca8853a1001c1f4R70

On Fri, 31 Mar 2023 at 07:08, Jacek Lewandowski
 wrote:
>
> I'll do
>
> - - -- --- -  -
> Jacek Lewandowski
>
>
> czw., 30 mar 2023 o 22:09 Miklosovic, Stefan  
> napisał(a):
>>
>> Hi list,
>>
>> we are looking for one more committer to take a look at this patch (1, 2).
>>
>> It looks like there is a lot to go through because of number of files 
>> modified (around 200) but changes are really just about moving everything to 
>> CassandraRelevantProperties. I do not think that it should take more than 1 
>> hour of dedicated effort and we are done!
>>
>> Thanks in advance to whoever reviews this.
>>
>> I want to especially thank Maxim for his perseverance in this matter and I 
>> hope we will eventually deliver this work to trunk.
>>
>> (1) https://github.com/apache/cassandra/pull/2046
>> (2) https://issues.apache.org/jira/browse/CASSANDRA-17797
>>
>> Regards
>>
>> Regards
>>
>> 
>> From: Miklosovic, Stefan 
>> Sent: Wednesday, March 22, 2023 14:34
>> To: dev@cassandra.apache.org
>> Subject: Re: [DISCUSS] Moving system property names to the 
>> CassandraRelevantProperties
>>
>> NetApp Security WARNING: This is an external email. Do not click links or 
>> open attachments unless you recognize the sender and know the content is 
>> safe.
>>
>>
>>
>>
>> Hi Maxim,
>>
>> thanks for letting us know.
>>
>> I reviewed it couple

Re: [DISCUSS] Feature branch version hygiene

[Mick Semb Wever]

So when a CEP slips, do we have to create a 5.1-cep-N?
>


No, you'd just rename it, easy to do in just one place.
I really don't care, but the version would at least helps indicate what the
branch is getting rebased off.




> My personal view is that 5.0 should not be used for any resolved tickets -
> they should go to 5.0-alpha1, since this is the correct release for them.
> 5.0 can then be the target version, which makes more sense given it isn’t a
> concrete release.
>


Each time, we don't know if the first release will be an alpha1 or if we're
confident enough to go straight to a beta1.
A goal with stable trunk would make the latter possible.

And with the additional 5.0 label has been requested by a few to make it
easy to search for what's new, this has been the simplest way.

Re: [DISCUSS] Feature branch version hygiene

[Josh McKenzie]

> My personal view is that 5.0 should not be used for any resolved tickets - 
> they should go to 5.0-alpha1, since this is the correct release for them. 5.0 
> can then be the target version, which makes more sense given it isn’t a 
> concrete release.
Well now you're just opening Pandora's box about our strange idioms with 
FixVersion usage. ;)

> every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty 
> clear what this means.
I think this diverges from our current paradigm where "5.x" == next feature 
release, "5.0.x" == next patch release (i.e. bugfix only). Not to say it's bad, 
just an adjustment... which if we're open to adjustment...

I'm receptive to transitioning the discussion to that either on this thread or 
another; IMO we remain in a strange and convoluted place with our 
FixVersioning. My understanding of our current practice:
 • .x is used to denote target version. For example: 5.x, 5.0.x, 5.1.x, 4.1.x
 • When a ticket is committed, the FixVersion is transitioned to resolve the X 
to the next unreleased version in which it'll release
 • Weird Things are done to make this work for the release process and release 
manager on feature releases (alpha, beta, etc)
 • There's no clear fit for feature branch tickets in the above schema

And if I take what I think you're proposing here and extrapolate it out:
 • .0 is used to denote target version. For example: 5.0. 5.0.0. 5.1.0. 4.1.0
 • This appears to break down for patch releases: we _do_ release .0 versions 
of them rather than alpha/beta/etc, so a ticket targeting 4.1.0 would initially 
mean 2 different things based on resolved vs. unresolved status (resolved == in 
release, unresolved == targeting next unreleased) and that distinction would 
disappear on resolution (i.e. resolved + 4.1.0 would no longer definitively 
mean "contained in .0 release")
 • When a release is cut, we bulk update FixVersion ending in .0 to the release 
version in which they're contained (not clear how to disambiguate the things 
from the above bullet point)
 • For feature releases, .0 will transition to -alpha1
One possible solution would be to just no longer release a .0 version of things 
and reserve .0 to indicate "parked". I don't particularly like that but it's 
not the worst.

Another possible solution would be to just scrap this approach entirely and go 
with:
 • FixVersion on unreleased _and still advocated for tickets_ always targets 
the next unreleased version. For other tickets where nobody is advocating for 
their work / inclusion, we either FixVersion "Backlog" or close as "Later"
 • When a release is cut, roll all unresolved tickets w/that FixVersion to the 
next unreleased FixVersion
 • When we're gearing up to a release, we can do a broad pass on everything 
that's unreleased w/the next feature releases FixVersion and move tickets that 
are desirable but not blockers to the next unreleased FixVersion (patch for 
bug, minor/major for improvements or new features)
 • CEP tickets target the same FixVersion (i.e. next unreleased Feature 
release) as their parents. When the parent epic gets a new FixVersion on 
resolution, all children get that FixVersion (i.e. when we merge the CEP and 
update its FixVersion, we bulk update all children tickets)

On Thu, May 18, 2023, at 9:08 AM, Benedict wrote:
> 
> I don’t think we should over complicate this with special CEP release 
> targets. If we do, they shouldn’t be versioned.
> 
> My personal view is that 5.0 should not be used for any resolved tickets - 
> they should go to 5.0-alpha1, since this is the correct release for them. 5.0 
> can then be the target version, which makes more sense given it isn’t a 
> concrete release.
> 
> But, in lieu of that, every ticket targeting 5.0 could use fixVersion 5.0.x, 
> since it is pretty clear what this means. Some tickets that don’t hit 5.0.0 
> can then be postponed to a later version, but it’s not like this is 
> burdensome. Anything marked feature/improvement and 5.0.x gets bumped to 
> 5.1.x.
> 
> 
> 
> 
> 
> 
>> On 18 May 2023, at 13:58, Josh McKenzie  wrote:
>> 
>> CEP-N seems like a good compromise. NextMajorRelease bumps into our 
>> interchangeable use of "Major" and "Minor" from a semver perspective and 
>> could get confusing. Suppose we could do NextFeatureRelease, but at that 
>> point why not just have it linked to the CEP and have the epic set.
>> 
>> On Thu, May 18, 2023, at 12:26 AM, Caleb Rackliffe wrote:
>>> ...otherwise I'm fine w/ just the CEP name, like "CEP-7" for SAI, etc.
>>> 
>>> On Wed, May 17, 2023 at 11:24 PM Caleb Rackliffe  
>>> wrote:
 So when a CEP slips, do we have to create a 5.1-cep-N? Could we just have 
 a version that's "NextMajorRelease" or something like that? It should 
 still be pretty easy to bulk replace if we have something else to filter 
 on, like belonging to an epic?
 
 On Wed, May 17, 2023 at 6:42 PM Mick Semb Wever  wrote:
> 
> 
> On Tue, 16 May 2023 at 13:02, 

Re: [DISCUSS] Feature branch version hygiene

[Benedict]

So we just rename alpha1 to beta1 if that happens?

Or, we point resolved tickets straight to 5.0.0, and add 5.0-alpha1 to any 
tickets with *only* 5.0.0

This is probably the easiest for folk to understand when browsing.

Finding new features is easy either way - look for 5.0.0.

> On 18 May 2023, at 15:08, Mick Semb Wever  wrote:
> 
> 
> 
> 
>> So when a CEP slips, do we have to create a 5.1-cep-N? 
> 
> 
> No, you'd just rename it, easy to do in just one place.
> I really don't care, but the version would at least helps indicate what the 
> branch is getting rebased off.
> 
> 
>  
>> My personal view is that 5.0 should not be used for any resolved tickets - 
>> they should go to 5.0-alpha1, since this is the correct release for them. 
>> 5.0 can then be the target version, which makes more sense given it isn’t a 
>> concrete release.
> 
> 
> Each time, we don't know if the first release will be an alpha1 or if we're 
> confident enough to go straight to a beta1.
> A goal with stable trunk would make the latter possible.
> 
> And with the additional 5.0 label has been requested by a few to make it easy 
> to search for what's new, this has been the simplest way.
> 

Re: [DISCUSS] Feature branch version hygiene

[Benedict]

The .x approach only breaks down for unreleased majors, for which all of our intuitions breakdown and we rehash it every year.My mental model, though, is that anything that’s not a concrete release number is a target version. Which is where 5.0 goes wrong - it’s not a release so it should be a target, but for some reason we use it as a placeholder to park work arriving in 5.0.0.If we instead use 5.0.0 for this purpose, we just need to get 5.0-alpha1 labels added when those releases are cut.Then I propose we break the confusion in both directions by scrapping 5.0 entirely and introducing 5.0-target.So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are resolved (with additional labels as necessary)Simples?On 18 May 2023, at 15:21, Josh McKenzie  wrote:My personal view is that 5.0 should not be used for any resolved tickets - they should go to 5.0-alpha1, since this is the correct release for them. 5.0 can then be the target version, which makes more sense given it isn’t a concrete release.Well now you're just opening Pandora's box about our strange idioms with FixVersion usage. ;)every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty clear what this means.I think this diverges from our current paradigm where "5.x" == next feature release, "5.0.x" == next patch release (i.e. bugfix only). Not to say it's bad, just an adjustment... which if we're open to adjustment...I'm receptive to transitioning the discussion to that either on this thread or another; IMO we remain in a strange and convoluted place with our FixVersioning. My understanding of our current practice:.x is used to denote target version. For example: 5.x, 5.0.x, 5.1.x, 4.1.xWhen a ticket is committed, the FixVersion is transitioned to resolve the X to the next unreleased version in which it'll releaseWeird Things are done to make this work for the release process and release manager on feature releases (alpha, beta, etc)There's no clear fit for feature branch tickets in the above schemaAnd if I take what I think you're proposing here and extrapolate it out:.0 is used to denote target version. For example: 5.0. 5.0.0. 5.1.0. 4.1.0This appears to break down for patch releases: we _do_ release .0 versions of them rather than alpha/beta/etc, so a ticket targeting 4.1.0 would initially mean 2 different things based on resolved vs. unresolved status (resolved == in release, unresolved == targeting next unreleased) and that distinction would disappear on resolution (i.e. resolved + 4.1.0 would no longer definitively mean "contained in .0 release")When a release is cut, we bulk update FixVersion ending in .0 to the release version in which they're contained (not clear how to disambiguate the things from the above bullet point)For feature releases, .0 will transition to -alpha1One possible solution would be to just no longer release a .0 version of things and reserve .0 to indicate "parked". I don't particularly like that but it's not the worst.Another possible solution would be to just scrap this approach entirely and go with:FixVersion on unreleased _and still advocated for tickets_ always targets the next unreleased version. For other tickets where nobody is advocating for their work / inclusion, we either FixVersion "Backlog" or close as "Later"When a release is cut, roll all unresolved tickets w/that FixVersion to the next unreleased FixVersionWhen we're gearing up to a release, we can do a broad pass on everything that's unreleased w/the next feature releases FixVersion and move tickets that are desirable but not blockers to the next unreleased FixVersion (patch for bug, minor/major for improvements or new features)CEP tickets target the same FixVersion (i.e. next unreleased Feature release) as their parents. When the parent epic gets a new FixVersion on resolution, all children get that FixVersion (i.e. when we merge the CEP and update its FixVersion, we bulk update all children tickets)On Thu, May 18, 2023, at 9:08 AM, Benedict wrote:I don’t think we should over complicate this with special CEP release targets. If we do, they shouldn’t be versioned.My personal view is that 5.0 should not be used for any resolved tickets - they should go to 5.0-alpha1, since this is the correct release for them. 5.0 can then be the target version, which makes more sense given it isn’t a concrete release.But, in lieu of that, every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty clear what this means. Some tickets that don’t hit 5.0.0 can then be postponed to a later version, but it’s not like this is burdensome. Anything marked feature/improvement and 5.0.x gets bumped to 5.1.x.On 18 May 2023, at 13:58, Josh McKenzie  wrote:CEP-N seems like a good compromise. NextMajorRelease bumps into our interchangeable use of "Major" and "Minor" from a semver perspective and could get confusing. Suppose we could do NextFeatureRelease, but at that point why not just have it linked to the CEP and have the epic set.On Th

Re: [DISCUSS] Feature branch version hygiene

[Josh McKenzie]

> My mental model, though, is that anything that’s not a concrete release 
> number is a target version. Which is where 5.0 goes wrong - it’s not a 
> release so it should be a target, but for some reason we use it as a 
> placeholder to park work arriving in 5.0.0.
Ahhh.

> So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are 
> resolved (with additional labels as necessary)
Adding -target would definitely make things more clear. If we moved to "5.0 == 
unreleased, always move to something on commit" then you still have to find 
some external source to figure out what's going on w/our versioning.

I like 5.0-target. Easy to query for "FixVersion = 5.0-target AND type != 
'Bug'" to find stragglers after GA is cut to move to 5.1-target.

Still have the "update children FixVersion for feature branch when branch is 
merged" bit but that's not so onerous.

On Thu, May 18, 2023, at 10:28 AM, Benedict wrote:
> 
> The .x approach only breaks down for unreleased majors, for which all of our 
> intuitions breakdown and we rehash it every year.
> 
> My mental model, though, is that anything that’s not a concrete release 
> number is a target version. Which is where 5.0 goes wrong - it’s not a 
> release so it should be a target, but for some reason we use it as a 
> placeholder to park work arriving in 5.0.0.
> 
> If we instead use 5.0.0 for this purpose, we just need to get 5.0-alpha1 
> labels added when those releases are cut.
> 
> Then I propose we break the confusion in both directions by scrapping 5.0 
> entirely and introducing 5.0-target.
> 
> So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are 
> resolved (with additional labels as necessary)
> 
> Simples?
> 
>> On 18 May 2023, at 15:21, Josh McKenzie  wrote:
>> 
>>> My personal view is that 5.0 should not be used for any resolved tickets - 
>>> they should go to 5.0-alpha1, since this is the correct release for them. 
>>> 5.0 can then be the target version, which makes more sense given it isn’t a 
>>> concrete release.
>> Well now you're just opening Pandora's box about our strange idioms with 
>> FixVersion usage. ;)
>> 
>>> every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty 
>>> clear what this means.
>> I think this diverges from our current paradigm where "5.x" == next feature 
>> release, "5.0.x" == next patch release (i.e. bugfix only). Not to say it's 
>> bad, just an adjustment... which if we're open to adjustment...
>> 
>> I'm receptive to transitioning the discussion to that either on this thread 
>> or another; IMO we remain in a strange and convoluted place with our 
>> FixVersioning. My understanding of our current practice:
>>  • .x is used to denote target version. For example: 5.x, 5.0.x, 5.1.x, 4.1.x
>>  • When a ticket is committed, the FixVersion is transitioned to resolve the 
>> X to the next unreleased version in which it'll release
>>  • Weird Things are done to make this work for the release process and 
>> release manager on feature releases (alpha, beta, etc)
>>  • There's no clear fit for feature branch tickets in the above schema
>> 
>> And if I take what I think you're proposing here and extrapolate it out:
>>  • .0 is used to denote target version. For example: 5.0. 5.0.0. 5.1.0. 4.1.0
>>  • This appears to break down for patch releases: we _do_ release .0 
>> versions of them rather than alpha/beta/etc, so a ticket targeting 4.1.0 
>> would initially mean 2 different things based on resolved vs. unresolved 
>> status (resolved == in release, unresolved == targeting next unreleased) and 
>> that distinction would disappear on resolution (i.e. resolved + 4.1.0 would 
>> no longer definitively mean "contained in .0 release")
>>  • When a release is cut, we bulk update FixVersion ending in .0 to the 
>> release version in which they're contained (not clear how to disambiguate 
>> the things from the above bullet point)
>>  • For feature releases, .0 will transition to -alpha1
>> One possible solution would be to just no longer release a .0 version of 
>> things and reserve .0 to indicate "parked". I don't particularly like that 
>> but it's not the worst.
>> 
>> Another possible solution would be to just scrap this approach entirely and 
>> go with:
>>  • FixVersion on unreleased _and still advocated for tickets_ always targets 
>> the next unreleased version. For other tickets where nobody is advocating 
>> for their work / inclusion, we either FixVersion "Backlog" or close as 
>> "Later"
>>  • When a release is cut, roll all unresolved tickets w/that FixVersion to 
>> the next unreleased FixVersion
>>  • When we're gearing up to a release, we can do a broad pass on everything 
>> that's unreleased w/the next feature releases FixVersion and move tickets 
>> that are desirable but not blockers to the next unreleased FixVersion (patch 
>> for bug, minor/major for improvements or new features)
>>  • CEP tickets target the same FixVersion (i.e. next unrelea

Re: [DISCUSS] Feature branch version hygiene

[Jeremiah D Jordan]

So what do we do with feature branch merged tickets in this model?  They stay 
on 5.0-target after close and move to 5.0.0 when the epic is merged and closes?

> On May 18, 2023, at 9:33 AM, Josh McKenzie  wrote:
> 
>> My mental model, though, is that anything that’s not a concrete release 
>> number is a target version. Which is where 5.0 goes wrong - it’s not a 
>> release so it should be a target, but for some reason we use it as a 
>> placeholder to park work arriving in 5.0.0.
> Ahhh.
> 
>> So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are 
>> resolved (with additional labels as necessary)
> Adding -target would definitely make things more clear. If we moved to "5.0 
> == unreleased, always move to something on commit" then you still have to 
> find some external source to figure out what's going on w/our versioning.
> 
> I like 5.0-target. Easy to query for "FixVersion = 5.0-target AND type != 
> 'Bug'" to find stragglers after GA is cut to move to 5.1-target.
> 
> Still have the "update children FixVersion for feature branch when branch is 
> merged" bit but that's not so onerous.
> 
> On Thu, May 18, 2023, at 10:28 AM, Benedict wrote:
>> 
>> The .x approach only breaks down for unreleased majors, for which all of our 
>> intuitions breakdown and we rehash it every year.
>> 
>> My mental model, though, is that anything that’s not a concrete release 
>> number is a target version. Which is where 5.0 goes wrong - it’s not a 
>> release so it should be a target, but for some reason we use it as a 
>> placeholder to park work arriving in 5.0.0.
>> 
>> If we instead use 5.0.0 for this purpose, we just need to get 5.0-alpha1 
>> labels added when those releases are cut.
>> 
>> Then I propose we break the confusion in both directions by scrapping 5.0 
>> entirely and introducing 5.0-target.
>> 
>> So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are 
>> resolved (with additional labels as necessary)
>> 
>> Simples?
>> 
>>> On 18 May 2023, at 15:21, Josh McKenzie  wrote:
>>> 
 My personal view is that 5.0 should not be used for any resolved tickets - 
 they should go to 5.0-alpha1, since this is the correct release for them. 
 5.0 can then be the target version, which makes more sense given it isn’t 
 a concrete release.
>>> Well now you're just opening Pandora's box about our strange idioms with 
>>> FixVersion usage. ;)
>>> 
 every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty 
 clear what this means.
>>> I think this diverges from our current paradigm where "5.x" == next feature 
>>> release, "5.0.x" == next patch release (i.e. bugfix only). Not to say it's 
>>> bad, just an adjustment... which if we're open to adjustment...
>>> 
>>> I'm receptive to transitioning the discussion to that either on this thread 
>>> or another; IMO we remain in a strange and convoluted place with our 
>>> FixVersioning. My understanding of our current practice:
>>> .x is used to denote target version. For example: 5.x, 5.0.x, 5.1.x, 4.1.x
>>> When a ticket is committed, the FixVersion is transitioned to resolve the X 
>>> to the next unreleased version in which it'll release
>>> Weird Things are done to make this work for the release process and release 
>>> manager on feature releases (alpha, beta, etc)
>>> There's no clear fit for feature branch tickets in the above schema
>>> 
>>> And if I take what I think you're proposing here and extrapolate it out:
>>> .0 is used to denote target version. For example: 5.0. 5.0.0. 5.1.0. 4.1.0
>>> This appears to break down for patch releases: we _do_ release .0 versions 
>>> of them rather than alpha/beta/etc, so a ticket targeting 4.1.0 would 
>>> initially mean 2 different things based on resolved vs. unresolved status 
>>> (resolved == in release, unresolved == targeting next unreleased) and that 
>>> distinction would disappear on resolution (i.e. resolved + 4.1.0 would no 
>>> longer definitively mean "contained in .0 release")
>>> When a release is cut, we bulk update FixVersion ending in .0 to the 
>>> release version in which they're contained (not clear how to disambiguate 
>>> the things from the above bullet point)
>>> For feature releases, .0 will transition to -alpha1
>>> One possible solution would be to just no longer release a .0 version of 
>>> things and reserve .0 to indicate "parked". I don't particularly like that 
>>> but it's not the worst.
>>> 
>>> Another possible solution would be to just scrap this approach entirely and 
>>> go with:
>>> FixVersion on unreleased _and still advocated for tickets_ always targets 
>>> the next unreleased version. For other tickets where nobody is advocating 
>>> for their work / inclusion, we either FixVersion "Backlog" or close as 
>>> "Later"
>>> When a release is cut, roll all unresolved tickets w/that FixVersion to the 
>>> next unreleased FixVersion
>>> When we're gearing up to a release, we can do a broad pass on everyth

Re: [DISCUSS] Feature branch version hygiene

[Josh McKenzie]

> They stay on 5.0-target after close and move to 5.0.0 when the epic is merged 
> and closes
Yep. When they merge to the feature branch they're still not on trunk (or 
whatever release branch) so they're still targeting it.

That leaves us with the one indicative case: something with "resolution = Fixed 
AND FixVersion ~ 'target'" means it's on a feature branch. At time of feature 
branch merge folks will need to update the FixVersion on the epic + all child 
tickets that are merged onto that branch.

On Thu, May 18, 2023, at 11:11 AM, Jeremiah D Jordan wrote:
> So what do we do with feature branch merged tickets in this model?  *They 
> stay on 5.0-target after close and move to 5.0.0 when the epic is merged and 
> closes*?
> 
>> On May 18, 2023, at 9:33 AM, Josh McKenzie  wrote:
>> 
>>> My mental model, though, is that anything that’s not a concrete release 
>>> number is a target version. Which is where 5.0 goes wrong - it’s not a 
>>> release so it should be a target, but for some reason we use it as a 
>>> placeholder to park work arriving in 5.0.0.
>> Ahhh.
>> 
>>> So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are 
>>> resolved (with additional labels as necessary)
>> Adding -target would definitely make things more clear. If we moved to "5.0 
>> == unreleased, always move to something on commit" then you still have to 
>> find some external source to figure out what's going on w/our versioning.
>> 
>> I like 5.0-target. Easy to query for "FixVersion = 5.0-target AND type != 
>> 'Bug'" to find stragglers after GA is cut to move to 5.1-target.
>> 
>> Still have the "update children FixVersion for feature branch when branch is 
>> merged" bit but that's not so onerous.
>> 
>> On Thu, May 18, 2023, at 10:28 AM, Benedict wrote:
>>> 
>>> The .x approach only breaks down for unreleased majors, for which all of 
>>> our intuitions breakdown and we rehash it every year.
>>> 
>>> My mental model, though, is that anything that’s not a concrete release 
>>> number is a target version. Which is where 5.0 goes wrong - it’s not a 
>>> release so it should be a target, but for some reason we use it as a 
>>> placeholder to park work arriving in 5.0.0.
>>> 
>>> If we instead use 5.0.0 for this purpose, we just need to get 5.0-alpha1 
>>> labels added when those releases are cut.
>>> 
>>> Then I propose we break the confusion in both directions by scrapping 5.0 
>>> entirely and introducing 5.0-target.
>>> 
>>> So tickets go to 5.0-target if they target 5.0, and to 5.0.0 once they are 
>>> resolved (with additional labels as necessary)
>>> 
>>> Simples?
>>> 
 On 18 May 2023, at 15:21, Josh McKenzie  wrote:
 
> My personal view is that 5.0 should not be used for any resolved tickets 
> - they should go to 5.0-alpha1, since this is the correct release for 
> them. 5.0 can then be the target version, which makes more sense given it 
> isn’t a concrete release.
 Well now you're just opening Pandora's box about our strange idioms with 
 FixVersion usage. ;)
 
> every ticket targeting 5.0 could use fixVersion 5.0.x, since it is pretty 
> clear what this means.
 I think this diverges from our current paradigm where "5.x" == next 
 feature release, "5.0.x" == next patch release (i.e. bugfix only). Not to 
 say it's bad, just an adjustment... which if we're open to adjustment...
 
 I'm receptive to transitioning the discussion to that either on this 
 thread or another; IMO we remain in a strange and convoluted place with 
 our FixVersioning. My understanding of our current practice:
  • .x is used to denote target version. For example: 5.x, 5.0.x, 5.1.x, 
 4.1.x
  • When a ticket is committed, the FixVersion is transitioned to resolve 
 the X to the next unreleased version in which it'll release
  • Weird Things are done to make this work for the release process and 
 release manager on feature releases (alpha, beta, etc)
  • There's no clear fit for feature branch tickets in the above schema
 
 And if I take what I think you're proposing here and extrapolate it out:
  • .0 is used to denote target version. For example: 5.0. 5.0.0. 5.1.0. 
 4.1.0
  • This appears to break down for patch releases: we _do_ release .0 
 versions of them rather than alpha/beta/etc, so a ticket targeting 4.1.0 
 would initially mean 2 different things based on resolved vs. unresolved 
 status (resolved == in release, unresolved == targeting next unreleased) 
 and that distinction would disappear on resolution (i.e. resolved + 4.1.0 
 would no longer definitively mean "contained in .0 release")
  • When a release is cut, we bulk update FixVersion ending in .0 to the 
 release version in which they're contained (not clear how to disambiguate 
 the things from the above bullet point)
  • For feature releases, .0 will transition to -alpha1
 One possible solut