Re: Immediately Deprecated Code

2023-11-01 Thread Claude Warren, Jr via dev
My thought was that we have code that is intended to be used for a specific
time frame.  We should clean up the code base when that code is no longer
used.  But we don't have any good way to track that.  This proposal was an
attempt to provide signposts for removing such code.

On Tue, Oct 31, 2023 at 6:56 PM Mick Semb Wever  wrote:

>
> For online upgrades we support skipping majors so long as the majors are
> adjacent.
> That is, any 4.x.z to any 5.x.z
>  ( Is it recommended that you always first patch upgrade the .z to the
> latest before the major upgrade. )
>
> For offline upgrades, we are aiming to maintain all compatibility.
>
> Take care when removing code, there are various (serdes) classes that look
> like they are for other components but are also used in the storage engine.
>
>
>
> On Tue, 31 Oct 2023 at 18:42, Claude Warren, Jr via dev <
> dev@cassandra.apache.org> wrote:
>
>> In your example 5.1 can read 4.x because 4.0 (?) is the earliest version
>> that 5.x supports.  I don't think you can upgrade directly from 3.x to 5.x
>> without an intermediate stop at some version of 4.x can you?  So when we
>> get to 6.x we won't need the 4 -> 5 conversion code because 6 will only
>> support reading from  5.  If I am incorrect and we expect a version to be
>> able to read 2 major versions back then indeed the deprecated since would
>> be 2 major versions ahead of the version when the code was written.
>>
>> On Tue, Oct 31, 2023 at 2:40 PM Andrew Weaver 
>> wrote:
>>
>>> Skipping versions on upgrade is absolutely something that happens in the
>>> real world.  This is particularly highlighted by the discussion around
>>> 5.0/5.1 that's been happening - 5.0 has been described as a potential
>>> "ghost version" which I completely understand.
>>>
>>> Getting rid of some of the old cruft that seems unnecessary (and
>>> strictly speaking is unnecessary) is not without its downsides.  In this
>>> case, that cruft improves the user experience.
>>>
>>> On Tue, Oct 31, 2023 at 5:56 AM Miklosovic, Stefan via dev <
>>> dev@cassandra.apache.org> wrote:
>>>
 Do I understand it correctly that this is basically the case of
 "deprecated on introduction" as we know that it will not be necessary the
 very next version?

 I think that not everybody is upgrading from version to version as they
 appear. If somebody upgrades from 4.0 to 5.1 (which we seem to support) (1)
 and you would have introduced the deprecation in 4.0 with intention to
 remove it in 5.0 and somebody jumps to 5.1 straight away, is not that a
 problem? Because they have not made the move via 5.0 where you upgrade
 logic was triggered.

 (1)
 https://github.com/apache/cassandra/blob/trunk/test/distributed/org/apache/cassandra/distributed/upgrade/UpgradeTestBase.java#L97-L108

 
 From: Claude Warren, Jr via dev 
 Sent: Tuesday, October 31, 2023 10:57
 To: dev
 Cc: Claude Warren, Jr
 Subject: Immediately Deprecated Code

 NetApp Security WARNING: This is an external email. Do not click links
 or open attachments unless you recognize the sender and know the content is
 safe.



 I was thinking about code that is used to migrate from one version to
 another.  For example the code that rewrote the order of the hash values
 used for Bloom filters.  That code was necessary for the version it was
 coded in.  But the next version does not need that code because the next
 version is not going to read the data from 2 versions prior to itself.  So
 the code could be removed for verson+1.

 So, would it have made sense to annotate those methods (and variables)
 as deprecated since the version they were written in so the
 methods/variables can be removed in the next version?

 If so, what I propose is that all transitional methods and variable be
 marked as deprecated with the version in which they were introduced.


>>>
>>> --
>>> Andrew Weaver
>>>
>>


Re: Immediately Deprecated Code

2023-11-01 Thread Mick Semb Wever
> My thought was that we have code that is intended to be used for a
> specific time frame.
>


With your example of bloom filters, if that's on-disk then it is not to be
removed.

Specifically, code that is used in relation to a sstable format still
listed in BigFormat.BigVersion (or BtiFormat.BtiVersion) must be kept.  And
there are no plans to remove it.  See CASSANDRA-18312

There's been some deprecated since annotations added recently that imply a
version of Cassandra is using a specific sstable format version.  This is
no longer true (see the  storage_compatibility_mode yaml option).  We are
to consider past sstable formats are still in use in newer clusters.
Though, the introduction of the option bound the lowest format version to
'nb' (see CASSANDRA-14227).


Re: [CMWG] November 1 meeting

2023-11-01 Thread Melissa Logan
CMWG meeting starts in 20 minutes.

Join via Zoom:
*https://us02web.zoom.us/j/82210868338?pwd=V3hrV3BUd2duVU5mVkE4RWhBNDZ3Zz09*


On Mon, Oct 30, 2023 at 12:12 PM Melissa Logan 
wrote:

> Hi everyone,
>
> Please join us this Wednesday, November 1 for the monthly Cassandra
> Working Group (CMWG) meeting at 8am PT / 11 ET /  3pm UTC.
>
> Add your updates or new discussions/questions to the doc:
> https://docs.google.com/document/d/1_73-iSjo1ZKKFn410rOCqNctnhNJOFLvbVA225OdrXY/edit
>
> If you need access to the doc, just request it (or hit reply and I will
> add).
>
> Draft agenda
>
>-
>
>Cassandra Summit 
>+ AI.Dev 
>(Patrick)
>-
>
>Catalyst program update (Melissa)
>-
>
>Meetup panel suggestions for Cassandra + AI panel (Melissa)
>-
>
>November 28 Contributor Meeting (Melissa)
>-
>
>Cassandra marketing (Melissa)
>
>
> Speak soon!
>
> Melissa
>


Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Josh McKenzie
> That vote thread also did not reach the threshold; it was incorrectly 
> counted, as committer votes are not binding for procedural changes. I counted 
> at most 8 PMC +1 votes. 
This piqued my curiosity.

Link to how we vote: 
https://cwiki.apache.org/confluence/display/CASSANDRA/Cassandra+Project+Governance
*STATUS: Ratified 2020/06/25*

Relevant bits here:
> On dev@:
> 
>  1. Discussion / binding votes on releases (Consensus: min 3 PMC +1, no -1)
>  2. Discussion / binding votes on project structure and governance changes 
> (adopting subprojects, how we vote and govern, etc). (super majority)

The thread where we voted on the CI bar Jeremiah referenced: 
https://lists.apache.org/thread/2shht9rb0l8fh2gfqx6sz9pxobo6sr60
Particularly relevant bit:
> Committer / pmc votes binding. Simple majority passes.
I think you're arguing that voting to change our bar for merging when it comes 
to CI falls under "votes on project structure"? I think when I called that vote 
I was conceptualizing it as a technical discussion about a shared norm on how 
we as committers deal with code contributions, where the "committer votes are 
binding, simple majority" applies.

I can see credible arguments in either direction, though I'd have expected 
those concerns or counter-arguments to have come up back in Jan of 2022 when we 
voted on the CI changes, not almost 2 years later after us operating under this 
new shared norm. The sentiments expressed on the discuss and vote thread were 
consistently positive and uncontentious; this feels to me like it falls 
squarely under the spirit of lazy consensus only at a much larger buy-in level 
than usual: 
https://community.apache.org/committers/decisionMaking.html#lazy-consensus

We've had plenty of time to call this vote and merge bar into question (i.e. 
every ticket we merge we're facing the "no regressions" bar), and the only 
reason I'd see us treating TCM or Accord differently would be because they're 
much larger bodies of work at merge so it's going to be a bigger lift to get to 
non-regression CI, and/or we would want a release cut from a formal branch 
rather than a feature branch for preview.

An alternative approach to keep this merge and CI burden lower would have been 
more incremental work merged into trunk periodically, an argument many folks in 
the community have made in the past. I personally have mixed feelings about it; 
there's pros and cons to both approaches.

All that said, I'm in favor of us continuing with this as a valid and ratified 
vote (technical norms == committer binding + simple majority). If we want to 
open a formal discussion about instead considering that a procedural change and 
rolling things back based on those grounds I'm fine with that, but we'll need 
to discuss that and think about the broader implications since things like 
changing import ordering, tooling, or other ecosystem-wide impacting changes 
(CI systems we all share, etc) would similarly potentially run afoul of needing 
supermajority pmc participation of we categorize that type of work as "project 
structure" as per the governance rules.

On Tue, Oct 31, 2023, at 1:25 PM, Jeremy Hanna wrote:
> I think the goal is to say "how could we get some working version of 
> TCM/Accord into people's hands to try out at/by Summit?"  That's all.  People 
> are eager to see it and try it out.
> 
>> On Oct 31, 2023, at 12:16 PM, Benedict  wrote:
>> 
>> 
>> No, if I understand it correctly we’re in weird hypothetical land where 
>> people are inventing new release types (“preview”) to avoid merging TCM[1] 
>> in the event we want to cut a 5.1 release from the PR prior to the summit if 
>> there’s some handful of failing tests in the PR. 
>> 
>> This may or may not be a waste of everyone’s time.
>> 
>> Jeremiah, I’m not questioning: it was procedurally invalid. How we handle 
>> that is, as always, a matter for community decision making.
>> 
>> [1] how this helps isn’t entirely clear
>> 
>> 
>>> On 31 Oct 2023, at 17:08, Paulo Motta  wrote:
>>> 
>>> Even if it was not formally prescribed as far as I understand, we have been 
>>> following the "only merge on Green CI" custom as much as possible for the 
>>> past several years. Is the proposal to relax this rule for 5.0?
>>> 
>>> On Tue, Oct 31, 2023 at 1:02 PM Jeremiah Jordan  
>>> wrote:
 You are free to argue validity.  I am just stating what I see on the 
 mailing list and in the wiki.  We had a vote which was called passing and 
 was not contested at that time.  The vote was on a process which includes 
 as #3 in the list:
 
  1. Before a merge, a committer needs either a non-regressing (i.e. no new 
 failures) run of circleci with the required test suites (TBD; see below) 
 or of ci-cassandra.
1. Non-regressing is defined here as "Doesn't introduce any new test 
 failures; any new failures in CI are clearly not attributable to this diff"
2. (NEW) After merging tickets, ci-cassandra runs a

Re: Development Dependencies documentation.

2023-11-01 Thread Abe Ratnofsky
Following back up here - patch has been updated and is ready for review: 
https://github.com/apache/cassandra-website/pull/170

> On Oct 25, 2023, at 8:07 AM, Ekaterina Dimitrova  
> wrote:
> 
> Hi Claude,
> You are not wrong. Unfortunately, it is outdated. Abe Ratnofsky has a work in 
> progress patch. You might want to get in touch with him to finish it.
> Best regards,
> Ekaterina
> 
> On Wed, 25 Oct 2023 at 8:04, Claude Warren, Jr via dev 
> mailto:dev@cassandra.apache.org>> wrote:
>> I just had to change dependencies in Cassandra for the first  time and I 
>> think the documentation [1] is out of date.
>> 
>> First I think most of the file edits are in the ".build" directory.  Adding 
>> jars to the "lib" directory works until calling "ant realclean", so perhaps 
>> the instructions should include regenerating the "lib" folder after making 
>> the edits.
>> 
>> If I am wrong please let me know, otherwise I will open a ticket and update 
>> the documentation.
>> 
>> [1] https://cassandra.apache.org/_/development/dependencies.html



Re: [CMWG] November 1 meeting

2023-11-01 Thread Melissa Logan
Meeting notes and recording are below. Patrick shared awesome keynotes that
will be at Cassandra Summit. We're in the final stretch for event
promotions, so please share far and wide. You can use posts in the social
doc link below to promote Cassandra Summit, community events, video content
and other C updates.

Our last meeting of the year is Wednesday, December 6 at 8:00 AM PT. See
you then!

Meeting Notes:
https://cwiki.apache.org/confluence/display/CASSANDRA/2023-10-04++Meeting

Recording:
https://us02web.zoom.us/rec/share/rzDtDev6ZUAyl34khsY6uLhu_4UkonGP_JqY0qhoZrRlSroBBlHpcDR2yYrRXW3m.ejGlWmZHGJ5G7rA7
Passcode: ?ngYU3W^

Social Posts:
https://docs.google.com/document/d/1VhoblApDzR6o4F0PNyjZTeQqizQtJjTPsgtEJeof-X0/edit#heading=h.3qzwljmzv9zs


On Wed, Nov 1, 2023 at 7:42 AM Melissa Logan  wrote:

> CMWG meeting starts in 20 minutes.
>
> Join via Zoom:
> *https://us02web.zoom.us/j/82210868338?pwd=V3hrV3BUd2duVU5mVkE4RWhBNDZ3Zz09*
> 
>
> On Mon, Oct 30, 2023 at 12:12 PM Melissa Logan 
> wrote:
>
>> Hi everyone,
>>
>> Please join us this Wednesday, November 1 for the monthly Cassandra
>> Working Group (CMWG) meeting at 8am PT / 11 ET /  3pm UTC.
>>
>> Add your updates or new discussions/questions to the doc:
>> https://docs.google.com/document/d/1_73-iSjo1ZKKFn410rOCqNctnhNJOFLvbVA225OdrXY/edit
>>
>> If you need access to the doc, just request it (or hit reply and I will
>> add).
>>
>> Draft agenda
>>
>>-
>>
>>Cassandra Summit
>> + AI.Dev
>> (Patrick)
>>-
>>
>>Catalyst program update (Melissa)
>>-
>>
>>Meetup panel suggestions for Cassandra + AI panel (Melissa)
>>-
>>
>>November 28 Contributor Meeting (Melissa)
>>-
>>
>>Cassandra marketing (Melissa)
>>
>>
>> Speak soon!
>>
>> Melissa
>>
>


Re: [CMWG] November 1 meeting

2023-11-01 Thread Melissa Logan
Here are correct links for this month's recording and meeting notes.

Meeting Notes:
https://cwiki.apache.org/confluence/display/CASSANDRA/2023-11-01

Recording:
https://us02web.zoom.us/rec/share/B9nFjAjUPhIpy1rf6ZaXWK_quWAakMM48gkfIyf1gKqFbqQiIWi72YDnqt9YwFSf.HqxDJ50FJk1BIUG7
Passcode: ^XA=$Nx6

On Wed, Nov 1, 2023 at 9:14 AM Melissa Logan  wrote:

> Meeting notes and recording are below. Patrick shared awesome keynotes
> that will be at Cassandra Summit. We're in the final stretch for event
> promotions, so please share far and wide. You can use posts in the social
> doc link below to promote Cassandra Summit, community events, video content
> and other C updates.
>
> Our last meeting of the year is Wednesday, December 6 at 8:00 AM PT. See
> you then!
>
> Meeting Notes:
> https://cwiki.apache.org/confluence/display/CASSANDRA/2023-10-04++Meeting
>
> Recording:
> https://us02web.zoom.us/rec/share/rzDtDev6ZUAyl34khsY6uLhu_4UkonGP_JqY0qhoZrRlSroBBlHpcDR2yYrRXW3m.ejGlWmZHGJ5G7rA7
> Passcode: ?ngYU3W^
>
> Social Posts:
> https://docs.google.com/document/d/1VhoblApDzR6o4F0PNyjZTeQqizQtJjTPsgtEJeof-X0/edit#heading=h.3qzwljmzv9zs
>
>
> On Wed, Nov 1, 2023 at 7:42 AM Melissa Logan 
> wrote:
>
>> CMWG meeting starts in 20 minutes.
>>
>> Join via Zoom:
>> *https://us02web.zoom.us/j/82210868338?pwd=V3hrV3BUd2duVU5mVkE4RWhBNDZ3Zz09*
>> 
>>
>> On Mon, Oct 30, 2023 at 12:12 PM Melissa Logan 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> Please join us this Wednesday, November 1 for the monthly Cassandra
>>> Working Group (CMWG) meeting at 8am PT / 11 ET /  3pm UTC.
>>>
>>> Add your updates or new discussions/questions to the doc:
>>> https://docs.google.com/document/d/1_73-iSjo1ZKKFn410rOCqNctnhNJOFLvbVA225OdrXY/edit
>>>
>>> If you need access to the doc, just request it (or hit reply and I will
>>> add).
>>>
>>> Draft agenda
>>>
>>>-
>>>
>>>Cassandra Summit
>>> + AI.Dev
>>> (Patrick)
>>>-
>>>
>>>Catalyst program update (Melissa)
>>>-
>>>
>>>Meetup panel suggestions for Cassandra + AI panel (Melissa)
>>>-
>>>
>>>November 28 Contributor Meeting (Melissa)
>>>-
>>>
>>>Cassandra marketing (Melissa)
>>>
>>>
>>> Speak soon!
>>>
>>> Melissa
>>>
>>


Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Benedict
Your conceptualisation implies no weight to the decision, as a norm is not binding?The community voting section mentions only three kinds of decision, and this was deliberate: code contributions, CEP and releases - the latter of which non-PMC members are only permitted to veto; their votes do not count positively[1]. Everything else is a PMC decision.> I think you're arguing that voting to change our bar for merging when it comes to CI falls under "votes on project structure”“Votes on project structure and governance”. Governance, per Wikipedia, is "the way rules, norms and actions are structured and sustained.”I do not see any ambiguity here. The community side provides no basis for a vote of this kind, while the PMC side specifically reserves this kind of decision. But evidently we need to make this clearer.Regarding the legitimacy of questioning this now: I have not come up against this legislation before. The norm of requiring green CI has been around for a lot longer than this vote, so nothing much changed until we started questioning the specifics of this legislation. At this point, the legitimacy of the decision also matters. Clearly there is broad support for a policy of this kind, but is this specific policy adequate?While I endorse the general sentiment of the policy, I do not endorse a policy that has no wiggle room. I have made every effort in all of my policy-making to ensure there are loosely-defined escape hatches for the community to use, in large part to minimise this kind of legalistic logjam, which is just wasted cycles.On 1 Nov 2023, at 15:31, Josh McKenzie  wrote:That vote thread also did not reach the threshold; it was incorrectly counted, as committer votes are not binding for procedural changes. I counted at most 8 PMC +1 votes. This piqued my curiosity.Link to how we vote: https://cwiki.apache.org/confluence/display/CASSANDRA/Cassandra+Project+GovernanceSTATUS: Ratified 2020/06/25Relevant bits here:On dev@:Discussion / binding votes on releases (Consensus: min 3 PMC +1, no -1)Discussion / binding votes on project structure and governance changes (adopting subprojects, how we vote and govern, etc). (super majority)The thread where we voted on the CI bar Jeremiah referenced: https://lists.apache.org/thread/2shht9rb0l8fh2gfqx6sz9pxobo6sr60Particularly relevant bit:Committer / pmc votes binding.
Simple majority passes.I think you're arguing that voting to change our bar for merging when it comes to CI falls under "votes on project structure"? I think when I called that vote I was conceptualizing it as a technical discussion about a shared norm on how we as committers deal with code contributions, where the "committer votes are binding, simple majority" applies.I can see credible arguments in either direction, though I'd have expected those concerns or counter-arguments to have come up back in Jan of 2022 when we voted on the CI changes, not almost 2 years later after us operating under this new shared norm. The sentiments expressed on the discuss and vote thread were consistently positive and uncontentious; this feels to me like it falls squarely under the spirit of lazy consensus only at a much larger buy-in level than usual: https://community.apache.org/committers/decisionMaking.html#lazy-consensusWe've had plenty of time to call this vote and merge bar into question (i.e. every ticket we merge we're facing the "no regressions" bar), and the only reason I'd see us treating TCM or Accord differently would be because they're much larger bodies of work at merge so it's going to be a bigger lift to get to non-regression CI, and/or we would want a release cut from a formal branch rather than a feature branch for preview.An alternative approach to keep this merge and CI burden lower would have been more incremental work merged into trunk periodically, an argument many folks in the community have made in the past. I personally have mixed feelings about it; there's pros and cons to both approaches.All that said, I'm in favor of us continuing with this as a valid and ratified vote (technical norms == committer binding + simple majority). If we want to open a formal discussion about instead considering that a procedural change and rolling things back based on those grounds I'm fine with that, but we'll need to discuss that and think about the broader implications since things like changing import ordering, tooling, or other ecosystem-wide impacting changes (CI systems we all share, etc) would similarly potentially run afoul of needing supermajority pmc participation of we categorize that type of work as "project structure" as per the governance rules.On Tue, Oct 31, 2023, at 1:25 PM, Jeremy Hanna wrote:I think the goal is to say "how could we get some working version of TCM/Accord into people's hands to try out at/by Summit?"  That's all.  People are eager to see it and try it out.On Oct 31, 2023, at 12:16 PM, Benedict  wrote:No, if I understand it correctly we’re in weird hypothetical land w

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Josh McKenzie
First off, I appreciate your time and attention on this stuff. Want to be up 
front about that since these kinds of discussions can get prickly all too 
easily. I'm *at least* as guilty as anyone else about getting my back up on 
stuff like this. Figuring out the right things to "harden" as shared 
contractual ways we behave and what to leave loose and case-by-case is going to 
continue to be a challenge for us as we grow.

The last thing I personally want is for us to have too many extraneous rules 
formalizing things that just serve to slow down peoples' ability to contribute 
to the project. The flip side of that - for all of us to work in a shared space 
and collectively remain maximally productive, some individual freedoms (ability 
to merge a bunch of broken code and/or ninja in things as we see fit, needing 2 
committers' eyes on things, etc) will have to be given up.

At it's core the discussion we had was prompted by divergence between circle 
and ASF CI and our release process dragging on repeatedly during the "stabilize 
ASF CI" phase. The "do we require green ci before merge of tickets" seems like 
it came along as an intuitive rider; best I can recall my thinking was "how 
else could we have a manageable load to stabilize in ASF CI if we didn't even 
require green circle before merging things in", but we didn't really dig into 
details; from a re-reading now, that portion of the discussion was just taken 
for granted as us being in alignment. Given it was a codifying a norm and 
everyone else in the discussion generally agreed, I don't think I or anyone 
thought to question it.


> “Votes on project structure *and governance”*. Governance, per Wikipedia, is 
> "the way rules, norms and actions are structured and sustained.”
> 
Bluntly, I'm not that worried about what wikipedia or a dictionary says about 
the topic. What I'm worried about here is what *we collectively as a community* 
think of as governance. "Do we have green CI pre-merge or not", *to me 
personally*, didn't qualify as a governance issue but rather a technical 
change. I'm open to being convinced otherwise, that things like that should 
qualify for a higher bar of voting, but again, I'm leery of that slowing down 
other workflow optimizations, changes, or community-wide impacting improvements 
people are making. Or muddying the waters to where people aren't sure what does 
or doesn't qualify as governance so they end up not pursuing things they're 
interested in improving as they're off-put by the bureaucratic burden of 
getting supermajority buy-in from pmc members who are much harder to get to 
participate in discussions on the ML compared to showing up for roll-call. 
:innocent:

My understanding of "The Apache Way" is that to move at speed and at scale, we 
need to trust each other to do the right thing and know we can back out if 
things go awry. So if some folks talk through mutating config through virtual 
tables for instance, or folks working on TCM put things up for review and I 
don't have cycles, I trust the folks doing that work (the committers working on 
it or review it) that I personally just stay out of it knowing that if things 
need refining going forward we'll do so. Different things have a different cost 
to refine and/or back out, sure, but in this case it's discussions + wiki 
articles. Not too painful.

So in this case here - is there a formal counter to that side of the discussion 
you'd want to open now? i.e. the case for merging bodies of work without green 
CI, when we do that, how we do that, why we do that? We very well could have 
missed a very meaningful and useful scenario that would have changed the 
collective conversation since nobody brought it up at the time. We simple 
majority committer voted in; we can simple majority committer vote out if we 
think this is too constricting a policy or if we want to add an exception to it 
right?

That's the blessing and the curse of decisions made with a lower bar; lower bar 
to undo.

And I suppose secondly - if you disagree on whether something qualifies for the 
super majority governance bar vs. the simple majority committer bar... how do 
we navigate that?

On Wed, Nov 1, 2023, at 12:33 PM, Benedict wrote:
> 
> 
> 
> Your conceptualisation implies no weight to the decision, as a norm is not 
> binding?
> 
> 
> 
> The community voting section mentions only three kinds of decision, and this 
> was deliberate: code contributions, CEP and releases - the latter of which 
> non-PMC members are only permitted to veto; their votes do not count 
> positively[1]. Everything else is a PMC decision.
> 
> 
> 
> > I think you're arguing that voting to change our bar for merging when it 
> > comes to CI falls under "votes on project structure”
> 
> 
> 
> “Votes on project structure *and governance”*. Governance, per Wikipedia, is 
> "the way rules, norms and actions are structured and sustained.”
> 
> 
> 
> I do not see any ambiguity here. The community side provid

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Benedict
The project governance document does not list any kind of general purpose technical change vote. There are only three very specific kinds of community vote: code contributions, CEP and release votes.  Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise. I suggest carefully re-reading the document we both formulated!If it is a technical contribution, as you contest, we only need a normal technical contribution vote to override it - i.e. two committer +1s. If that’s how we want to roll with it, I guess we’re not really in disagreement.None of this really fundamentally changes anything. There’s a strong norm for a commit gate on CI, and nobody is going to go about breaking this norm willy-nilly. But equally there’s no need to panic and waste all this time debating hypothetical mechanisms to avoid this supposedly ironclad rule.We clearly need to address confusion over governance though. The idea that agreeing things carefully costs us agility is one I cannot endorse. The project has leaned heavily into the consensus side of the Apache Way, as evidenced by our governance document. That doesn’t mean things can’t change quickly, it just means before those changes become formal requirements there needs to be broad consensus, as defined in the governing document. That’s it.The norm existed before the vote, and it exists whether the vote was valid or not. That is how things evolve on the project, we just formalise them a little more slowly.On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:First off, I appreciate your time and attention on this stuff. Want to be up front about that since these kinds of discussions can get prickly all too easily. I'm at least as guilty as anyone else about getting my back up on stuff like this. Figuring out the right things to "harden" as shared contractual ways we behave and what to leave loose and case-by-case is going to continue to be a challenge for us as we grow.The last thing I personally want is for us to have too many extraneous rules formalizing things that just serve to slow down peoples' ability to contribute to the project. The flip side of that - for all of us to work in a shared space and collectively remain maximally productive, some individual freedoms (ability to merge a bunch of broken code and/or ninja in things as we see fit, needing 2 committers' eyes on things, etc) will have to be given up.At it's core the discussion we had was prompted by divergence between circle and ASF CI and our release process dragging on repeatedly during the "stabilize ASF CI" phase. The "do we require green ci before merge of tickets" seems like it came along as an intuitive rider; best I can recall my thinking was "how else could we have a manageable load to stabilize in ASF CI if we didn't even require green circle before merging things in", but we didn't really dig into details; from a re-reading now, that portion of the discussion was just taken for granted as us being in alignment. Given it was a codifying a norm and everyone else in the discussion generally agreed, I don't think I or anyone thought to question it.“Votes on project structure and governance”. Governance, per Wikipedia, is "the way rules, norms and actions are structured and sustained.”Bluntly, I'm not that worried about what wikipedia or a dictionary says about the topic. What I'm worried about here is what we collectively as a community think of as governance. "Do we have green CI pre-merge or not", to me personally, didn't qualify as a governance issue but rather a technical change. I'm open to being convinced otherwise, that things like that should qualify for a higher bar of voting, but again, I'm leery of that slowing down other workflow optimizations, changes, or community-wide impacting improvements people are making. Or muddying the waters to where people aren't sure what does or doesn't qualify as governance so they end up not pursuing things they're interested in improving as they're off-put by the bureaucratic burden of getting supermajority buy-in from pmc members who are much harder to get to participate in discussions on the ML compared to showing up for roll-call. :innocent:My understanding of "The Apache Way" is that to move at speed and at scale, we need to trust each other to do the right thing and know we can back out if things go awry. So if some folks talk through mutating config through virtual tables for instance, or folks working on TCM put things up for review and I don't have cycles, I trust the folks doing that work (the committers working on it or review it) that I personally just stay out of it knowing that if things need refining going forward we'll do so. Different things have a different cost to refine and/or back out, sure, but in this case it's discussions + wiki articles. Not too painful.So in this case here - is there a formal counter to that side of the discussion you'd want to open now? i.e. the case fo

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Benedict
> The idea that agreeing things carefully costs us agility is one I cannot endorsenot one I can endorse 🙄On 1 Nov 2023, at 21:11, Benedict  wrote:The project governance document does not list any kind of general purpose technical change vote. There are only three very specific kinds of community vote: code contributions, CEP and release votes.  Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise. I suggest carefully re-reading the document we both formulated!If it is a technical contribution, as you contest, we only need a normal technical contribution vote to override it - i.e. two committer +1s. If that’s how we want to roll with it, I guess we’re not really in disagreement.None of this really fundamentally changes anything. There’s a strong norm for a commit gate on CI, and nobody is going to go about breaking this norm willy-nilly. But equally there’s no need to panic and waste all this time debating hypothetical mechanisms to avoid this supposedly ironclad rule.We clearly need to address confusion over governance though. The idea that agreeing things carefully costs us agility is one I cannot endorse. The project has leaned heavily into the consensus side of the Apache Way, as evidenced by our governance document. That doesn’t mean things can’t change quickly, it just means before those changes become formal requirements there needs to be broad consensus, as defined in the governing document. That’s it.The norm existed before the vote, and it exists whether the vote was valid or not. That is how things evolve on the project, we just formalise them a little more slowly.On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:First off, I appreciate your time and attention on this stuff. Want to be up front about that since these kinds of discussions can get prickly all too easily. I'm at least as guilty as anyone else about getting my back up on stuff like this. Figuring out the right things to "harden" as shared contractual ways we behave and what to leave loose and case-by-case is going to continue to be a challenge for us as we grow.The last thing I personally want is for us to have too many extraneous rules formalizing things that just serve to slow down peoples' ability to contribute to the project. The flip side of that - for all of us to work in a shared space and collectively remain maximally productive, some individual freedoms (ability to merge a bunch of broken code and/or ninja in things as we see fit, needing 2 committers' eyes on things, etc) will have to be given up.At it's core the discussion we had was prompted by divergence between circle and ASF CI and our release process dragging on repeatedly during the "stabilize ASF CI" phase. The "do we require green ci before merge of tickets" seems like it came along as an intuitive rider; best I can recall my thinking was "how else could we have a manageable load to stabilize in ASF CI if we didn't even require green circle before merging things in", but we didn't really dig into details; from a re-reading now, that portion of the discussion was just taken for granted as us being in alignment. Given it was a codifying a norm and everyone else in the discussion generally agreed, I don't think I or anyone thought to question it.“Votes on project structure and governance”. Governance, per Wikipedia, is "the way rules, norms and actions are structured and sustained.”Bluntly, I'm not that worried about what wikipedia or a dictionary says about the topic. What I'm worried about here is what we collectively as a community think of as governance. "Do we have green CI pre-merge or not", to me personally, didn't qualify as a governance issue but rather a technical change. I'm open to being convinced otherwise, that things like that should qualify for a higher bar of voting, but again, I'm leery of that slowing down other workflow optimizations, changes, or community-wide impacting improvements people are making. Or muddying the waters to where people aren't sure what does or doesn't qualify as governance so they end up not pursuing things they're interested in improving as they're off-put by the bureaucratic burden of getting supermajority buy-in from pmc members who are much harder to get to participate in discussions on the ML compared to showing up for roll-call. :innocent:My understanding of "The Apache Way" is that to move at speed and at scale, we need to trust each other to do the right thing and know we can back out if things go awry. So if some folks talk through mutating config through virtual tables for instance, or folks working on TCM put things up for review and I don't have cycles, I trust the folks doing that work (the committers working on it or review it) that I personally just stay out of it knowing that if things need refining going forward we'll do so. Different things have a different cost to refine and/or back out, sure, but in this case it's discussions + wiki

November community meetings with mondayDB, PMC chair, and 5.0

2023-11-01 Thread Melissa Logan
Join us for our virtual community meetings in November including:

   - Tuesday, Nov. 28: 5.0 Release Preview
   - Thursday, Nov. 30: mondayDB - Crafting a Database from Scratch with
   Liran Brimer + The State of the Cassandra Development Community with Josh
   McKenzie

Register: https://www.meetup.com/cassandra-global/

Yesterday's contributor meeting with Piotr Kołaczkowski on the CQL NOT
operator is now available: https://www.youtube.com/watch?v=HMA1usuU888

And here is last week's Cassandra case study with Patrick Lee of Walmart:
https://youtu.be/zIHkSLYVvMM

See you in a few weeks!

Melissa


Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Josh McKenzie
> Community voting is also entirely by consensus, there is no such thing as a 
> simple majority community vote, technical or otherwise.
Ah hah! You're absolutely correct in that this isn't one of our "blessed" ways 
we vote. There's nothing written down about "committers are binding, simple 
majority" for any specific category of discussion.

Are we ok with people creatively applying different ways to vote for things 
where there's not otherwise guidance if they feel it helps capture sentiment 
and engagement? Obviously the outcome of that isn't binding in the same way 
other votes by the pmc are, but binding to the same extent 2 committers 
reviewing something we later need to revert is binding.

I'd rather we have a bunch of committers weigh in if we're talking about 
changing import ordering, or Config.java structure, or refactoring out 
singletons, or gatekeeping CI - things we've had come up over the years where 
we've had a lot of people chime in and we benefit from more than just "2 
committers agree on it" but less than "We need a CEP or pmc vote for this".


On Wed, Nov 1, 2023, at 5:10 PM, Benedict wrote:
> 
> The project governance document does not list any kind of general purpose 
> technical change vote. There are only three very specific kinds of community 
> vote: code contributions, CEP and release votes.  Community voting is also 
> entirely by consensus, there is no such thing as a simple majority community 
> vote, technical or otherwise. I suggest carefully re-reading the document we 
> both formulated!
> 
> If it is a technical contribution, as you contest, we only need a normal 
> technical contribution vote to override it - i.e. two committer +1s. If 
> that’s how we want to roll with it, I guess we’re not really in disagreement.
> 
> None of this really fundamentally changes anything. There’s a strong norm for 
> a commit gate on CI, and nobody is going to go about breaking this norm 
> willy-nilly. But equally there’s no need to panic and waste all this time 
> debating hypothetical mechanisms to avoid this supposedly ironclad rule.
> 
> We clearly need to address confusion over governance though. The idea that 
> agreeing things carefully costs us agility is one I cannot endorse. The 
> project has leaned heavily into the consensus side of the Apache Way, as 
> evidenced by our governance document. That doesn’t mean things can’t change 
> quickly, it just means *before those changes become formal requirements 
> *there needs to be *broad* consensus, as defined in the governing document. 
> That’s it.
> 
> The norm existed before the vote, and it exists whether the vote was valid or 
> not. That is how things evolve on the project, we just formalise them a 
> little more slowly.
> 
> 
>> On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:
>> 
>> First off, I appreciate your time and attention on this stuff. Want to be up 
>> front about that since these kinds of discussions can get prickly all too 
>> easily. I'm *at least* as guilty as anyone else about getting my back up on 
>> stuff like this. Figuring out the right things to "harden" as shared 
>> contractual ways we behave and what to leave loose and case-by-case is going 
>> to continue to be a challenge for us as we grow.
>> 
>> The last thing I personally want is for us to have too many extraneous rules 
>> formalizing things that just serve to slow down peoples' ability to 
>> contribute to the project. The flip side of that - for all of us to work in 
>> a shared space and collectively remain maximally productive, some individual 
>> freedoms (ability to merge a bunch of broken code and/or ninja in things as 
>> we see fit, needing 2 committers' eyes on things, etc) will have to be given 
>> up.
>> 
>> At it's core the discussion we had was prompted by divergence between circle 
>> and ASF CI and our release process dragging on repeatedly during the 
>> "stabilize ASF CI" phase. The "do we require green ci before merge of 
>> tickets" seems like it came along as an intuitive rider; best I can recall 
>> my thinking was "how else could we have a manageable load to stabilize in 
>> ASF CI if we didn't even require green circle before merging things in", but 
>> we didn't really dig into details; from a re-reading now, that portion of 
>> the discussion was just taken for granted as us being in alignment. Given it 
>> was a codifying a norm and everyone else in the discussion generally agreed, 
>> I don't think I or anyone thought to question it.
>> 
>> 
>>> “Votes on project structure *and governance”*. Governance, per Wikipedia, 
>>> is "the way rules, norms and actions are structured and sustained.”
>>> 
>> Bluntly, I'm not that worried about what wikipedia or a dictionary says 
>> about the topic. What I'm worried about here is what *we collectively as a 
>> community* think of as governance. "Do we have green CI pre-merge or not", 
>> *to me personally*, didn't qualify as a governance issue but rather a 
>> technical chan

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Josh McKenzie
> but binding to the same extent 2 committers reviewing something we later need 
> to revert is binding.
To elaborate a bit - what I mean is "it's a bar we apply to help establish a 
baseline level of consensus but it's very much a 2-way door". Obviously 2 
committers +1'ing code is a formal agreed upon voting mechanism.

On Wed, Nov 1, 2023, at 7:26 PM, Josh McKenzie wrote:
>> Community voting is also entirely by consensus, there is no such thing as a 
>> simple majority community vote, technical or otherwise.
> Ah hah! You're absolutely correct in that this isn't one of our "blessed" 
> ways we vote. There's nothing written down about "committers are binding, 
> simple majority" for any specific category of discussion.
> 
> Are we ok with people creatively applying different ways to vote for things 
> where there's not otherwise guidance if they feel it helps capture sentiment 
> and engagement? Obviously the outcome of that isn't binding in the same way 
> other votes by the pmc are, but binding to the same extent 2 committers 
> reviewing something we later need to revert is binding.
> 
> I'd rather we have a bunch of committers weigh in if we're talking about 
> changing import ordering, or Config.java structure, or refactoring out 
> singletons, or gatekeeping CI - things we've had come up over the years where 
> we've had a lot of people chime in and we benefit from more than just "2 
> committers agree on it" but less than "We need a CEP or pmc vote for this".
> 
> 
> On Wed, Nov 1, 2023, at 5:10 PM, Benedict wrote:
>> 
>> The project governance document does not list any kind of general purpose 
>> technical change vote. There are only three very specific kinds of community 
>> vote: code contributions, CEP and release votes.  Community voting is also 
>> entirely by consensus, there is no such thing as a simple majority community 
>> vote, technical or otherwise. I suggest carefully re-reading the document we 
>> both formulated!
>> 
>> If it is a technical contribution, as you contest, we only need a normal 
>> technical contribution vote to override it - i.e. two committer +1s. If 
>> that’s how we want to roll with it, I guess we’re not really in disagreement.
>> 
>> None of this really fundamentally changes anything. There’s a strong norm 
>> for a commit gate on CI, and nobody is going to go about breaking this norm 
>> willy-nilly. But equally there’s no need to panic and waste all this time 
>> debating hypothetical mechanisms to avoid this supposedly ironclad rule.
>> 
>> We clearly need to address confusion over governance though. The idea that 
>> agreeing things carefully costs us agility is one I cannot endorse. The 
>> project has leaned heavily into the consensus side of the Apache Way, as 
>> evidenced by our governance document. That doesn’t mean things can’t change 
>> quickly, it just means *before those changes become formal requirements 
>> *there needs to be *broad* consensus, as defined in the governing document. 
>> That’s it.
>> 
>> The norm existed before the vote, and it exists whether the vote was valid 
>> or not. That is how things evolve on the project, we just formalise them a 
>> little more slowly.
>> 
>> 
>>> On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:
>>> 
>>> First off, I appreciate your time and attention on this stuff. Want to be 
>>> up front about that since these kinds of discussions can get prickly all 
>>> too easily. I'm *at least* as guilty as anyone else about getting my back 
>>> up on stuff like this. Figuring out the right things to "harden" as shared 
>>> contractual ways we behave and what to leave loose and case-by-case is 
>>> going to continue to be a challenge for us as we grow.
>>> 
>>> The last thing I personally want is for us to have too many extraneous 
>>> rules formalizing things that just serve to slow down peoples' ability to 
>>> contribute to the project. The flip side of that - for all of us to work in 
>>> a shared space and collectively remain maximally productive, some 
>>> individual freedoms (ability to merge a bunch of broken code and/or ninja 
>>> in things as we see fit, needing 2 committers' eyes on things, etc) will 
>>> have to be given up.
>>> 
>>> At it's core the discussion we had was prompted by divergence between 
>>> circle and ASF CI and our release process dragging on repeatedly during the 
>>> "stabilize ASF CI" phase. The "do we require green ci before merge of 
>>> tickets" seems like it came along as an intuitive rider; best I can recall 
>>> my thinking was "how else could we have a manageable load to stabilize in 
>>> ASF CI if we didn't even require green circle before merging things in", 
>>> but we didn't really dig into details; from a re-reading now, that portion 
>>> of the discussion was just taken for granted as us being in alignment. 
>>> Given it was a codifying a norm and everyone else in the discussion 
>>> generally agreed, I don't think I or anyone thought to question it.
>>> 
>>> 
 “

Re: Push TCM (CEP-21) and Accord (CEP-15) to 5.1 (and cut an immediate 5.1-alpha1)

2023-11-01 Thread Benedict
So my view is that the community is strongly built on consensus, so expressions of sentiment within the community have strong normative weight even without any specific legislative effect. You shouldn’t knowingly go against what appears to be a consensus (or even widely-held) view, even if it has no formal weight. So I’m not sure we need any additional mechanisms beyond DISCUSS threads, polls and lazy consensus?Let’s treat your thread as a POLL for arguments sake: lots of folk voted, and every vote was positive. So clearly there’s strong endorsement for the approach, or parts thereof, in some form. Given the goal of consensus in decision-making, it would not be reasonable to ignore this widely held view on contributions. This likely means at least another DISCUSS thread and lazy consensus if you want to knowingly go against it, or want to modify or clarify what’s meant. This just falls naturally out of how we do things here I think, and is how we go about a lot of business already. It retains the agility you were talking about, setting norms cheaply.It isn’t however a tightly held policy or legislative cudgel, it’s just what those who were talking and paying attention at the time agreed. It can be chucked out or rewoven at zero cost, but if the norms have taken hold and are broadly understood in the same way, it won’t change much or at all, because the actual glue is the norm, not the words, which only serve to broadcast some formulation of the norm.On 1 Nov 2023, at 23:41, Josh McKenzie  wrote:but binding to the same extent 2 committers reviewing something we later need to revert is binding.To elaborate a bit - what I mean is "it's a bar we apply to help establish a baseline level of consensus but it's very much a 2-way door". Obviously 2 committers +1'ing code is a formal agreed upon voting mechanism.On Wed, Nov 1, 2023, at 7:26 PM, Josh McKenzie wrote:Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise.Ah hah! You're absolutely correct in that this isn't one of our "blessed" ways we vote. There's nothing written down about "committers are binding, simple majority" for any specific category of discussion.Are we ok with people creatively applying different ways to vote for things where there's not otherwise guidance if they feel it helps capture sentiment and engagement? Obviously the outcome of that isn't binding in the same way other votes by the pmc are, but binding to the same extent 2 committers reviewing something we later need to revert is binding.I'd rather we have a bunch of committers weigh in if we're talking about changing import ordering, or Config.java structure, or refactoring out singletons, or gatekeeping CI - things we've had come up over the years where we've had a lot of people chime in and we benefit from more than just "2 committers agree on it" but less than "We need a CEP or pmc vote for this".On Wed, Nov 1, 2023, at 5:10 PM, Benedict wrote:The project governance document does not list any kind of general purpose technical change vote. There are only three very specific kinds of community vote: code contributions, CEP and release votes.  Community voting is also entirely by consensus, there is no such thing as a simple majority community vote, technical or otherwise. I suggest carefully re-reading the document we both formulated!If it is a technical contribution, as you contest, we only need a normal technical contribution vote to override it - i.e. two committer +1s. If that’s how we want to roll with it, I guess we’re not really in disagreement.None of this really fundamentally changes anything. There’s a strong norm for a commit gate on CI, and nobody is going to go about breaking this norm willy-nilly. But equally there’s no need to panic and waste all this time debating hypothetical mechanisms to avoid this supposedly ironclad rule.We clearly need to address confusion over governance though. The idea that agreeing things carefully costs us agility is one I cannot endorse. The project has leaned heavily into the consensus side of the Apache Way, as evidenced by our governance document. That doesn’t mean things can’t change quickly, it just means before those changes become formal requirements there needs to be broad consensus, as defined in the governing document. That’s it.The norm existed before the vote, and it exists whether the vote was valid or not. That is how things evolve on the project, we just formalise them a little more slowly.On 1 Nov 2023, at 20:07, Josh McKenzie  wrote:First off, I appreciate your time and attention on this stuff. Want to be up front about that since these kinds of discussions can get prickly all too easily. I'm at least as guilty as anyone else about getting my back up on stuff like this. Figuring out the right things to "harden" as shared contractual ways we behave and what to leave loose and case-by-case is going to continue to be a challenge for us as we grow.Th