Re: Downgradability

2023-02-23 Thread Claude Warren, Jr via dev 
Broken downgrading can be fixed (I think) by modifying the
SearializationHeader.toHeader() method where it currently throws an
UnknownColumnException.  If we can, instead of throwing the exception,
create a dropped column for the unexpected column then I think the code
will work.

I realise that to do this in the wild is not possible as it is a change to
released code, but we could handle it going forward.

On Wed, Feb 22, 2023 at 11:21 PM Henrik Ingo 
wrote:

> ... ok apparently shift+enter  sends messages now?
>
> I was just saying if at least the file format AND system/tables - anything
> written to disk - can be protected with a switch, then it allows for quick
> downgrade by shutting down the entire cluster and restarting with the
> downgraded binary. It's a start.
>
> To be able to do that live in a distributed system needs to consider much
> more: gossip, streaming, drivers, and ultimately all features, because we
> don't' want an application developer to use a shiny new thing that a) may
> not be available on all nodes, or b) may disappear if the cluster has to be
> downgraded later.
>
> henrik
>
> On Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo 
> wrote:
>
>> Just this once I'm going to be really brief :-)
>>
>> Just wanted to share for reference how Mongodb implemented
>> downgradeability around their 4.4 version:
>> https://www.mongodb.com/docs/manual/release-notes/6.0-downgrade-sharded-cluster/
>>
>> Jeff you're right. Ultimately this is about more than file formats.
>> However, ideally if at least the
>>
>> On Mon, Feb 20, 2023 at 10:02 PM Jeff Jirsa  wrote:
>>
>>> I'm not even convinced even 8110 addresses this - just writing sstables
>>> in old versions won't help if we ever add things like new types or new
>>> types of collections without other control abilities. Claude's other email
>>> in another thread a few hours ago talks about some of these surprises -
>>> "Specifically during the 3.1 -> 4.0 changes a column broadcast_port was
>>> added to system/local.  This means that 3.1 system can not read the table
>>> as it has no definition for it.  I tried marking the column for deletion in
>>> the metadata and in the serialization header.  The later got past the
>>> column not found problem, but I suspect that it just means that data
>>> columns after broadcast_port shifted and so incorrectly read." - this is a
>>> harder problem to solve than just versioning sstables and network
>>> protocols.
>>>
>>> Stepping back a bit, we have downgrade ability listed as a goal, but
>>> it's not (as far as I can tell) universally enforced, nor is it clear at
>>> which point we will be able to concretely say "this release can be
>>> downgraded to X".   Until we actually define and agree that this is a
>>> real goal with a concrete version where downgrade-ability becomes real, it
>>> feels like things are somewhat arbitrarily enforced, which is probably very
>>> frustrating for people trying to commit work/tickets.
>>>
>>> - Jeff
>>>
>>>
>>>
>>> On Mon, Feb 20, 2023 at 11:48 AM Dinesh Joshi  wrote:
>>>
 I’m a big fan of maintaining backward compatibility. Downgradability
 implies that we could potentially roll back an upgrade at any time. While I
 don’t think we need to retain the ability to downgrade in perpetuity it
 would be a good objective to maintain strict backward compatibility and
 therefore downgradability until a certain point. This would imply
 versioning metadata and extending it in such a way that prior version(s)
 could continue functioning. This can certainly be expensive to implement
 and might bloat on-disk storage. However, we could always offer an option
 for the operator to optimize the on-disk structures for the current version
 then we can rewrite them in the latest version. This optimizes the storage
 and opens up new functionality. This means new features that can work with
 old on-disk structures will be available while others that strictly require
 new versions of the data structures will be unavailable until the operator
 migrates to the new version. This migration IMO should be irreversible.
 Beyond this point the operator will lose the ability to downgrade which is
 ok.

 Dinesh

 On Feb 20, 2023, at 10:40 AM, Jake Luciani  wrote:

 
 There has been progress on

 https://issues.apache.org/jira/plugins/servlet/mobile#issue/CASSANDRA-8928

 Which is similar to what datastax does for DSE. Would this be an
 acceptable solution?

 Jake

 On Mon, Feb 20, 2023 at 11:17 AM guo Maxwell 
 wrote:

> It seems “An alternative solution is to implement/complete
> CASSANDRA-8110 ”
> can give us more options if it is finished😉
>
> Branimir Lambov 于2023年2月20日 周一下午11:03写道：
>
>> Hi everyone,
>>
>> There has been a discussion lately about changes to the sstable
>> format in t

Re: Downgradability

2023-02-23 Thread Benjamin Lerer 
>
> Can somebody explain to me what is so burdensome, that we seem to be
> spending longer debating it than it would take to implement the necessary
> changes?


I believe that we all agree on the outcome. Everybody wants
downgradability. The issue is on the path to get there.

As far as I am concerned, I would like to see a proper solution and as Jeff
suggested the equivalent of the upgrade tests as gatekeepers. Having
everybody trying to enforce it on his own way will only lead to a poor
result in my opinion with some addoc code that might not really guarantee
real downgradability in the end.
We have rushed in the past to get feature outs and pay the price for it. I
simply prefer that we take the time to do things right.

Thanks to Scott and you, downgradability got a much better visibility so no
matter what approach we pick, I am convinced that we will get there.

Le jeu. 23 févr. 2023 à 09:49, Claude Warren, Jr via dev <
dev@cassandra.apache.org> a écrit :

> Broken downgrading can be fixed (I think) by modifying the
> SearializationHeader.toHeader() method where it currently throws an
> UnknownColumnException.  If we can, instead of throwing the exception,
> create a dropped column for the unexpected column then I think the code
> will work.
>
> I realise that to do this in the wild is not possible as it is a change to
> released code, but we could handle it going forward.
>
> On Wed, Feb 22, 2023 at 11:21 PM Henrik Ingo 
> wrote:
>
>> ... ok apparently shift+enter  sends messages now?
>>
>> I was just saying if at least the file format AND system/tables -
>> anything written to disk - can be protected with a switch, then it allows
>> for quick downgrade by shutting down the entire cluster and restarting with
>> the downgraded binary. It's a start.
>>
>> To be able to do that live in a distributed system needs to consider much
>> more: gossip, streaming, drivers, and ultimately all features, because we
>> don't' want an application developer to use a shiny new thing that a) may
>> not be available on all nodes, or b) may disappear if the cluster has to be
>> downgraded later.
>>
>> henrik
>>
>> On Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo 
>> wrote:
>>
>>> Just this once I'm going to be really brief :-)
>>>
>>> Just wanted to share for reference how Mongodb implemented
>>> downgradeability around their 4.4 version:
>>> https://www.mongodb.com/docs/manual/release-notes/6.0-downgrade-sharded-cluster/
>>>
>>> Jeff you're right. Ultimately this is about more than file formats.
>>> However, ideally if at least the
>>>
>>> On Mon, Feb 20, 2023 at 10:02 PM Jeff Jirsa  wrote:
>>>
 I'm not even convinced even 8110 addresses this - just writing sstables
 in old versions won't help if we ever add things like new types or new
 types of collections without other control abilities. Claude's other email
 in another thread a few hours ago talks about some of these surprises -
 "Specifically during the 3.1 -> 4.0 changes a column broadcast_port was
 added to system/local.  This means that 3.1 system can not read the table
 as it has no definition for it.  I tried marking the column for deletion in
 the metadata and in the serialization header.  The later got past the
 column not found problem, but I suspect that it just means that data
 columns after broadcast_port shifted and so incorrectly read." - this is a
 harder problem to solve than just versioning sstables and network
 protocols.

 Stepping back a bit, we have downgrade ability listed as a goal, but
 it's not (as far as I can tell) universally enforced, nor is it clear at
 which point we will be able to concretely say "this release can be
 downgraded to X".   Until we actually define and agree that this is a
 real goal with a concrete version where downgrade-ability becomes real, it
 feels like things are somewhat arbitrarily enforced, which is probably very
 frustrating for people trying to commit work/tickets.

 - Jeff



 On Mon, Feb 20, 2023 at 11:48 AM Dinesh Joshi 
 wrote:

> I’m a big fan of maintaining backward compatibility. Downgradability
> implies that we could potentially roll back an upgrade at any time. While 
> I
> don’t think we need to retain the ability to downgrade in perpetuity it
> would be a good objective to maintain strict backward compatibility and
> therefore downgradability until a certain point. This would imply
> versioning metadata and extending it in such a way that prior version(s)
> could continue functioning. This can certainly be expensive to implement
> and might bloat on-disk storage. However, we could always offer an option
> for the operator to optimize the on-disk structures for the current 
> version
> then we can rewrite them in the latest version. This optimizes the storage
> and opens up new functionality. This means new features that can work with
> o

[DISCUSS] Single boilerplate script

2023-02-23 Thread Claude Warren, Jr via dev 
Pull request https://github.com/apache/cassandra/pull/1950/files is an
attempt to move the boilerplate coding from the script files into a single
maintainable file.

This change does 4 things:

   1. Moves the standard boiler plate from the standard scripts into a
   single maintainable script to be sourced in the original scripts with
   minimal changes.
   2. Creates a debug logging so that problem determination (as it
   CASSANDRA-17773 )
   is easier.
   3. Has code to preserve environment variables when needed (c.f. nodetool
   script).
   4. Provides a verification method that will verify that all standard
   environment variables are set.

In practice this is a simple 2 line replacement for most boilerplate
blocks.  Examples of the simple case (sstableloader) and a more complex
case (nodetool) are included in the current pull request.

If there is consensus, I will update the other scripts in the bin directory
to utilize the sourced boilerplate and request a full review of the pull
request.

Re: Downgradability

2023-02-23 Thread Benedict 
Forget downgradeability for a moment: we should not be breaking format compatibility without good reason. Bumping a major version isn’t enough of a reason. Can somebody explain to me why this is being fought tooth and nail, when the work involved is absolutely minimal?Regarding tests: what more do you want, than running our upgrade suite backwards?On 23 Feb 2023, at 09:45, Benjamin Lerer  wrote:
Can somebody explain to me what is so burdensome, that we seem to be 
spending longer debating it than it would take to implement the 
necessary changes?

I believe that we all agree on the outcome. Everybody wants downgradability. The issue is on the path to get there.As far as I am concerned, I would like to see a proper solution and as Jeff suggested the equivalent of the upgrade tests as gatekeepers. Having everybody trying to enforce it on his own way will only lead to a poor result in my opinion with some addoc code that might not really guarantee real downgradability in the end.We have rushed in the past to get feature outs and pay the price for it. I simply prefer that we take the time to do things right.Thanks to Scott and you, downgradability got a much better visibility so no matter what approach we pick, I am convinced that we will get there.Le jeu. 23 févr. 2023 à 09:49, Claude Warren, Jr via dev  a écrit :Broken downgrading can be fixed (I think) by modifying the SearializationHeader.toHeader() method where it currently throws an UnknownColumnException.  If we can, instead of throwing the exception, create a dropped column for the unexpected column then I think the code will work.I realise that to do this in the wild is not possible as it is a change to released code, but we could handle it going forward.On Wed, Feb 22, 2023 at 11:21 PM Henrik Ingo  wrote:... ok apparently shift+enter  sends messages now?I was just saying if at least the file format AND system/tables - anything written to disk - can be protected with a switch, then it allows for quick downgrade by shutting down the entire cluster and restarting with the downgraded binary. It's a start.To be able to do that live in a distributed system needs to consider much more: gossip, streaming, drivers, and ultimately all features, because we don't' want an application developer to use a shiny new thing that a) may not be available on all nodes, or b) may disappear if the cluster has to be downgraded later.henrikOn Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo  wrote:Just this once I'm going to be really brief :-)Just wanted to share for reference how Mongodb implemented downgradeability around their 4.4 version: https://www.mongodb.com/docs/manual/release-notes/6.0-downgrade-sharded-cluster/Jeff you're right. Ultimately this is about more than file formats. However, ideally if at least theOn Mon, Feb 20, 2023 at 10:02 PM Jeff Jirsa  wrote:I'm not even convinced even 8110 addresses this - just writing sstables in old versions won't help if we ever add things like new types or new types of collections without other control abilities. Claude's other email in another thread a few hours ago talks about some of these surprises - "Specifically during the 3.1 -> 4.0 changes a column broadcast_port was added to system/local.  This means that 3.1 system can not read the table as it has no definition for it.  I tried marking the column for deletion in the metadata and in the serialization header.  The later got past the column not found problem, but I suspect that it just means that data columns after broadcast_port shifted and so incorrectly read." - this is a harder problem to solve than just versioning sstables and network protocols. Stepping back a bit, we have downgrade ability listed as a goal, but it's not (as far as I can tell) universally enforced, nor is it clear at which point we will be able to concretely say "this release can be downgraded to X".   Until we actually define and agree that this is a real goal with a concrete version where downgrade-ability becomes real, it feels like things are somewhat arbitrarily enforced, which is probably very frustrating for people trying to commit work/tickets.- JeffOn Mon, Feb 20, 2023 at 11:48 AM Dinesh Joshi  wrote:I’m a big fan of maintaining backward compatibility. Downgradability implies that we could potentially roll back an upgrade at any time. While I don’t think we need to retain the ability to downgrade in perpetuity it would be a good objective to maintain strict backward compatibility and therefore downgradability until a certain point. This would imply versioning metadata and extending it in such a way that prior version(s) could continue functioning. This can certainly be expensive to implement and might bloat on-disk storage. However, we could always offer an option for the operator to optimize the on-disk structures for the current version then we can rewrite them in the latest ver

Re: Downgradability

2023-02-23 Thread Jacek Lewandowski 
Running upgrade tests backwards is great idea which does not require extra
work.

For stats metadata it already supports writing in previous serialization
version

We need a small fix in compression metadata and that's it.

A flag with the write format version is probably LHF.

Maybe let's try,  we still have time to fix it before 5.0


czw., 23 lut 2023, 10:57 użytkownik Benedict  napisał:

> Forget downgradeability for a moment: we should not be breaking format
> compatibility without good reason. Bumping a major version isn’t enough of
> a reason.
>
> Can somebody explain to me why this is being fought tooth and nail, when
> the work involved is absolutely minimal?
>
> Regarding tests: what more do you want, than running our upgrade suite
> backwards?
>
>
> On 23 Feb 2023, at 09:45, Benjamin Lerer  wrote:
>
> 
>
>> Can somebody explain to me what is so burdensome, that we seem to be
>> spending longer debating it than it would take to implement the necessary
>> changes?
>
>
> I believe that we all agree on the outcome. Everybody wants
> downgradability. The issue is on the path to get there.
>
> As far as I am concerned, I would like to see a proper solution and as
> Jeff suggested the equivalent of the upgrade tests as gatekeepers. Having
> everybody trying to enforce it on his own way will only lead to a poor
> result in my opinion with some addoc code that might not really guarantee
> real downgradability in the end.
> We have rushed in the past to get feature outs and pay the price for it. I
> simply prefer that we take the time to do things right.
>
> Thanks to Scott and you, downgradability got a much better visibility so
> no matter what approach we pick, I am convinced that we will get there.
>
> Le jeu. 23 févr. 2023 à 09:49, Claude Warren, Jr via dev <
> dev@cassandra.apache.org> a écrit :
>
>> Broken downgrading can be fixed (I think) by modifying the
>> SearializationHeader.toHeader() method where it currently throws an
>> UnknownColumnException.  If we can, instead of throwing the exception,
>> create a dropped column for the unexpected column then I think the code
>> will work.
>>
>> I realise that to do this in the wild is not possible as it is a
>> change to released code, but we could handle it going forward.
>>
>> On Wed, Feb 22, 2023 at 11:21 PM Henrik Ingo 
>> wrote:
>>
>>> ... ok apparently shift+enter  sends messages now?
>>>
>>> I was just saying if at least the file format AND system/tables -
>>> anything written to disk - can be protected with a switch, then it allows
>>> for quick downgrade by shutting down the entire cluster and restarting with
>>> the downgraded binary. It's a start.
>>>
>>> To be able to do that live in a distributed system needs to consider
>>> much more: gossip, streaming, drivers, and ultimately all features, because
>>> we don't' want an application developer to use a shiny new thing that a)
>>> may not be available on all nodes, or b) may disappear if the cluster has
>>> to be downgraded later.
>>>
>>> henrik
>>>
>>> On Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo 
>>> wrote:
>>>
 Just this once I'm going to be really brief :-)

 Just wanted to share for reference how Mongodb implemented
 downgradeability around their 4.4 version:
 https://www.mongodb.com/docs/manual/release-notes/6.0-downgrade-sharded-cluster/

 Jeff you're right. Ultimately this is about more than file formats.
 However, ideally if at least the

 On Mon, Feb 20, 2023 at 10:02 PM Jeff Jirsa  wrote:

> I'm not even convinced even 8110 addresses this - just writing
> sstables in old versions won't help if we ever add things like new types 
> or
> new types of collections without other control abilities. Claude's other
> email in another thread a few hours ago talks about some of these 
> surprises
> - "Specifically during the 3.1 -> 4.0 changes a column broadcast_port was
> added to system/local.  This means that 3.1 system can not read the table
> as it has no definition for it.  I tried marking the column for deletion 
> in
> the metadata and in the serialization header.  The later got past the
> column not found problem, but I suspect that it just means that data
> columns after broadcast_port shifted and so incorrectly read." - this is a
> harder problem to solve than just versioning sstables and network
> protocols.
>
> Stepping back a bit, we have downgrade ability listed as a goal, but
> it's not (as far as I can tell) universally enforced, nor is it clear at
> which point we will be able to concretely say "this release can be
> downgraded to X".   Until we actually define and agree that this is a
> real goal with a concrete version where downgrade-ability becomes real, it
> feels like things are somewhat arbitrarily enforced, which is probably 
> very
> frustrating for people trying to commit work/tickets.
>
> - Jeff
>
>
>
>>

Re: Downgradability

2023-02-23 Thread Claude Warren, Jr via dev 
You also need to remove the system.local.broadcast_port column as that does
not exist in the earlier version and when the earlier version attempts to
read it the code throws an UnknownColumnException.

On Thu, Feb 23, 2023 at 11:27 AM Jacek Lewandowski <
lewandowski.ja...@gmail.com> wrote:

> Running upgrade tests backwards is great idea which does not require extra
> work.
>
> For stats metadata it already supports writing in previous serialization
> version
>
> We need a small fix in compression metadata and that's it.
>
> A flag with the write format version is probably LHF.
>
> Maybe let's try,  we still have time to fix it before 5.0
>
>
> czw., 23 lut 2023, 10:57 użytkownik Benedict 
> napisał:
>
>> Forget downgradeability for a moment: we should not be breaking format
>> compatibility without good reason. Bumping a major version isn’t enough of
>> a reason.
>>
>> Can somebody explain to me why this is being fought tooth and nail, when
>> the work involved is absolutely minimal?
>>
>> Regarding tests: what more do you want, than running our upgrade suite
>> backwards?
>>
>>
>> On 23 Feb 2023, at 09:45, Benjamin Lerer  wrote:
>>
>> 
>>
>>> Can somebody explain to me what is so burdensome, that we seem to be
>>> spending longer debating it than it would take to implement the necessary
>>> changes?
>>
>>
>> I believe that we all agree on the outcome. Everybody wants
>> downgradability. The issue is on the path to get there.
>>
>> As far as I am concerned, I would like to see a proper solution and as
>> Jeff suggested the equivalent of the upgrade tests as gatekeepers. Having
>> everybody trying to enforce it on his own way will only lead to a poor
>> result in my opinion with some addoc code that might not really guarantee
>> real downgradability in the end.
>> We have rushed in the past to get feature outs and pay the price for it.
>> I simply prefer that we take the time to do things right.
>>
>> Thanks to Scott and you, downgradability got a much better visibility so
>> no matter what approach we pick, I am convinced that we will get there.
>>
>> Le jeu. 23 févr. 2023 à 09:49, Claude Warren, Jr via dev <
>> dev@cassandra.apache.org> a écrit :
>>
>>> Broken downgrading can be fixed (I think) by modifying the
>>> SearializationHeader.toHeader() method where it currently throws an
>>> UnknownColumnException.  If we can, instead of throwing the exception,
>>> create a dropped column for the unexpected column then I think the code
>>> will work.
>>>
>>> I realise that to do this in the wild is not possible as it is a
>>> change to released code, but we could handle it going forward.
>>>
>>> On Wed, Feb 22, 2023 at 11:21 PM Henrik Ingo 
>>> wrote:
>>>
 ... ok apparently shift+enter  sends messages now?

 I was just saying if at least the file format AND system/tables -
 anything written to disk - can be protected with a switch, then it allows
 for quick downgrade by shutting down the entire cluster and restarting with
 the downgraded binary. It's a start.

 To be able to do that live in a distributed system needs to consider
 much more: gossip, streaming, drivers, and ultimately all features, because
 we don't' want an application developer to use a shiny new thing that a)
 may not be available on all nodes, or b) may disappear if the cluster has
 to be downgraded later.

 henrik

 On Thu, Feb 23, 2023 at 1:14 AM Henrik Ingo 
 wrote:

> Just this once I'm going to be really brief :-)
>
> Just wanted to share for reference how Mongodb implemented
> downgradeability around their 4.4 version:
> https://www.mongodb.com/docs/manual/release-notes/6.0-downgrade-sharded-cluster/
>
> Jeff you're right. Ultimately this is about more than file formats.
> However, ideally if at least the
>
> On Mon, Feb 20, 2023 at 10:02 PM Jeff Jirsa  wrote:
>
>> I'm not even convinced even 8110 addresses this - just writing
>> sstables in old versions won't help if we ever add things like new types 
>> or
>> new types of collections without other control abilities. Claude's other
>> email in another thread a few hours ago talks about some of these 
>> surprises
>> - "Specifically during the 3.1 -> 4.0 changes a column broadcast_port was
>> added to system/local.  This means that 3.1 system can not read the table
>> as it has no definition for it.  I tried marking the column for deletion 
>> in
>> the metadata and in the serialization header.  The later got past the
>> column not found problem, but I suspect that it just means that data
>> columns after broadcast_port shifted and so incorrectly read." - this is 
>> a
>> harder problem to solve than just versioning sstables and network
>> protocols.
>>
>> Stepping back a bit, we have downgrade ability listed as a goal, but
>> it's not (as far as I can tell) universally enforced, nor is it clear

Re: Downgradability

2023-02-23 Thread Henrik Ingo 
On Thu, Feb 23, 2023 at 11:57 AM Benedict  wrote:

> Can somebody explain to me why this is being fought tooth and nail, when
> the work involved is absolutely minimal?
>
>
I don't know how each individual has been thinking about this, but it seems
to me just looking at all the tasks that at least the introduction of tries
is a major format change anyway - since it's the whole point - and
therefore people working on other tasks may have assumed the format is
changing anyway and therefore something like a switch (is this what is
referred to as the C-8110 solution?) will take care of it for everyone.

I'm not sure there's consensus that such a switch is a sufficient
resolution to this discussion, but if there were such a consensus, the next
question would be whether the patches that are otherwise ready now can
merge, or whether they will all be blocked waiting for the compatibility
solution first. And possibly better testing, etc. Letting them merge would
be justified by the desire to have more frequent and smaller increments of
work merged into trunk... well, I'm not going to repeat everything from
that discussion but the same pro's and con's apply.

henrik
-- 

Henrik Ingo

c. +358 40 569 7354

w. www.datastax.com

Re: Downgradability

2023-02-23 Thread Benedict 
I don’t think there’s anything about a new format that requires a version bump, but I could be missing something.We have to have a switch to enable tries already don’t we? I’m pretty sure we haven’t talked about switching the default format?On 23 Feb 2023, at 12:12, Henrik Ingo  wrote:On Thu, Feb 23, 2023 at 11:57 AM Benedict  wrote:Can somebody explain to me why this is being fought tooth and nail, when the work involved is absolutely minimal?I don't know how each individual has been thinking about this, but it seems to me just looking at all the tasks that at least the introduction of tries is a major format change anyway - since it's the whole point - and therefore people working on other tasks may have assumed the format is changing anyway and therefore something like a switch (is this what is referred to as the C-8110 solution?) will take care of it for everyone.I'm not sure there's consensus that such a switch is a sufficient resolution to this discussion, but if there were such a consensus, the next question would be whether the patches that are otherwise ready now can merge, or whether they will all be blocked waiting for the compatibility solution first. And possibly better testing, etc. Letting them merge would be justified by the desire to have more frequent and smaller increments of work merged into trunk... well, I'm not going to repeat everything from that discussion but the same pro's and con's apply.henrik -- Henrik Ingoc. +358 40 569 7354 w. www.datastax.com     

Re: Downgradability

2023-02-23 Thread Henrik Ingo 
Right. So I'm speculating everyone else who worked on a patch that breaks
compatibility has been working under the mindset "I'll just put this behind
the same switch". Or something more vague / even less correct, such as
assuming that tries would become the default immediately.

At least in my mind when we talk about the "switch to enable tries" I do
also consider things like "don't break streaming". So I guess whether one
considers "a switch" to exist already or not, might be subjective in this
case, because people have different assumptions on the definition of done
of such a switch.

henrik

On Thu, Feb 23, 2023 at 2:53 PM Benedict  wrote:

> I don’t think there’s anything about a new format that requires a version
> bump, but I could be missing something.
>
> We have to have a switch to enable tries already don’t we? I’m pretty sure
> we haven’t talked about switching the default format?
>
> On 23 Feb 2023, at 12:12, Henrik Ingo  wrote:
>
> 
> On Thu, Feb 23, 2023 at 11:57 AM Benedict  wrote:
>
>> Can somebody explain to me why this is being fought tooth and nail, when
>> the work involved is absolutely minimal?
>>
>>
> I don't know how each individual has been thinking about this, but it
> seems to me just looking at all the tasks that at least the introduction of
> tries is a major format change anyway - since it's the whole point - and
> therefore people working on other tasks may have assumed the format is
> changing anyway and therefore something like a switch (is this what is
> referred to as the C-8110 solution?) will take care of it for everyone.
>
> I'm not sure there's consensus that such a switch is a sufficient
> resolution to this discussion, but if there were such a consensus, the next
> question would be whether the patches that are otherwise ready now can
> merge, or whether they will all be blocked waiting for the compatibility
> solution first. And possibly better testing, etc. Letting them merge would
> be justified by the desire to have more frequent and smaller increments of
> work merged into trunk... well, I'm not going to repeat everything from
> that discussion but the same pro's and con's apply.
>
> henrik
> --
>
> Henrik Ingo
>
> c. +358 40 569 7354
>
> w. www.datastax.com
>
>
> 
> 
> 
> 
>
>

-- 

Henrik Ingo

c. +358 40 569 7354

w. www.datastax.com

Re: Downgradability

2023-02-23 Thread Benedict 
Either way, it feels like this has become much more of a big deal than it needed to.I would prefer the pending patches to avoid breaking compatibility, as I think they can do it easily. But, if we agree to block release until we can double back to fix it with versioned writing (which I agree with Jacek are LHF - I think we literally just need a method that chooses the descriptor version) then let’s not further agonise over this.Alternatively I’d be happy to work with the authors to get this done alongside their work, as I don’t think it would hold anything up. We just need something to pick a descriptor besides latest on write, everything else is basically there for these patches.On 23 Feb 2023, at 16:37, Henrik Ingo  wrote:Right. So I'm speculating everyone else who worked on a patch that breaks compatibility has been working under the mindset "I'll just put this behind the same switch". Or something more vague / even less correct, such as assuming that tries would become the default immediately.At least in my mind when we talk about the "switch to enable tries" I do also consider things like "don't break streaming". So I guess whether one considers "a switch" to exist already or not, might be subjective in this case, because people have different assumptions on the definition of done of such a switch.henrikOn Thu, Feb 23, 2023 at 2:53 PM Benedict  wrote:I don’t think there’s anything about a new format that requires a version bump, but I could be missing something.We have to have a switch to enable tries already don’t we? I’m pretty sure we haven’t talked about switching the default format?On 23 Feb 2023, at 12:12, Henrik Ingo  wrote:On Thu, Feb 23, 2023 at 11:57 AM Benedict  wrote:Can somebody explain to me why this is being fought tooth and nail, when the work involved is absolutely minimal?I don't know how each individual has been thinking about this, but it seems to me just looking at all the tasks that at least the introduction of tries is a major format change anyway - since it's the whole point - and therefore people working on other tasks may have assumed the format is changing anyway and therefore something like a switch (is this what is referred to as the C-8110 solution?) will take care of it for everyone.I'm not sure there's consensus that such a switch is a sufficient resolution to this discussion, but if there were such a consensus, the next question would be whether the patches that are otherwise ready now can merge, or whether they will all be blocked waiting for the compatibility solution first. And possibly better testing, etc. Letting them merge would be justified by the desire to have more frequent and smaller increments of work merged into trunk... well, I'm not going to repeat everything from that discussion but the same pro's and con's apply.henrik -- Henrik Ingoc. +358 40 569 7354 w. www.datastax.com      
-- Henrik Ingoc. +358 40 569 7354 w. www.datastax.com