Re: [DISCUSS] CEP-15: General Purpose Transactions

[bened...@apache.org]

Essentially this, although I think in practice we will need to track each 
partition’s timestamp separately (or optionally for reduced conflicts, each row 
or datum’s), and make them all part of the conditional application of the 
transaction - at least for strict-serializability.

The alternative is to insert read/write intents for the transaction during each 
step, and to confirm they are still valid on commit, but this approach would 
require a WAN round-trip for each step in the interactive transaction, whereas 
the timestamp-validating approach can use a LAN round-trip for each step 
besides the final one, and is also much simpler to implement.


From: Blake Eggleston 
Date: Thursday, 30 September 2021 at 05:47
To: dev@cassandra.apache.org 
Subject: Re: [DISCUSS] CEP-15: General Purpose Transactions
You could establish a lower timestamp bound and buffer transaction state on the 
coordinator, then make the commit an operation that only applies if all 
partitions involved haven’t been changed by a more recent timestamp. You could 
also implement mvcc either in the storage layer or for some period of time by 
buffering commits on each replica before applying.

> On Sep 29, 2021, at 6:18 PM, Jonathan Ellis  wrote:
>
> How are interactive transactions possible with Accord?
>
>
>
> On Tue, Sep 21, 2021 at 11:56 PM bened...@apache.org 
> wrote:
>
>> Could you explain why you believe this trade-off is necessary? We can
>> support full SQL just fine with Accord, and I hope that we eventually do so.
>>
>> This domain is incredibly complex, so it is easy to reach wrong
>> conclusions. I would invite you again to propose a system for discussion
>> that you think offers something Accord is unable to, and that you consider
>> desirable, and we can work from there.
>>
>> To pre-empt some possible discussions, I am not aware of anything we
>> cannot do with Accord that we could do with either Calvin or Spanner.
>> Interactive transactions are possible on top of Accord, as are transactions
>> with an unknown read/write set. In each case the only cost is that they
>> would use optimistic concurrency control, which is no worse the spanner
>> derivatives anyway (which I have to assume is your benchmark in this
>> regard). I do not expect to deliver either functionality initially, but
>> Accord takes us most of the way there for both.
>>
>>
>> From: Jonathan Ellis 
>> Date: Wednesday, 22 September 2021 at 05:36
>> To: dev 
>> Subject: Re: [DISCUSS] CEP-15: General Purpose Transactions
>> Right, I'm looking for exactly a discussion on the high level goals.
>> Instead of saying "here's the goals and we ruled out X because Y" we should
>> start with a discussion around, "Approach A allows X and W, approach B
>> allows Y and Z" and decide together what the goals should be and and what
>> we are willing to trade to get those goals, e.g., are we willing to give up
>> global strict serializability to get the ability to support full SQL.  Both
>> of these are nice to have!
>>
>> On Tue, Sep 21, 2021 at 9:52 PM bened...@apache.org 
>> wrote:
>>
>>> Hi Jonathan,
>>>
>>> These other systems are incompatible with the goals of the CEP. I do
>>> discuss them (besides 2PC) in both the whitepaper and the CEP, and will
>>> summarise that discussion below. A true and accurate comparison of these
>>> other systems is essentially intractable, as there are complex subtleties
>>> to each flavour, and those who are interested would be better served by
>>> performing their own research.
>>>
>>> I think it is more productive to focus on what we want to achieve as a
>>> community. If you believe the goals of this CEP are wrong for the
>> project,
>>> let’s focus on that. If you want to compare and contrast specific facets
>> of
>>> alternative systems that you consider to be preferable in some dimension,
>>> let’s do that here or in a Q&A as proposed by Joey.
>>>
>>> The relevant goals are that we:
>>>
>>>
>>>  1.  Guarantee strict serializable isolation on commodity hardware
>>>  2.  Scale to any cluster size
>>>  3.  Achieve optimal latency
>>>
>>> The approach taken by Spanner derivatives is rejected by (1) because they
>>> guarantee only Serializable isolation (they additionally fail (3)). From
>>> watching talks by YugaByte, and inferring from Cockroach’s
>>> panic-cluster-death under clock skew, this is clearly considered by
>>> everyone to be undesirable but necessary to achieve scalability.
>>>
>>> The approach taken by FaunaDB (Calvin) is rejected by (2) because its
>>> sequencing layer requires a global leader process for the cluster, which
>> is
>>> incompatible with Cassandra’s scalability requirements. It additionally
>>> fails (3) for global clients.
>>>
>>> Two phase commit fails (3). As an aside, AFAICT DynamoDB is today a
>>> Spanner clone for its multi-key transaction functionality, not 2PC.
>>>
>>> Systems such as RAMP with even weaker isolation are not considered for
>> the
>>> simple reason that they do not even claim to meet 

[DISCUSS] CEP-16: Auth Plugin Support for CQLSH

[bhouse99]

Hi everyone.  Are there any other thoughts or comments regarding the CEP-16: 
Auth Plugin Support for CQLSH proposal?

- 
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH

If not I'd like to send out a request for a vote.

Thanks!

Sent with ProtonMail Secure Email.

‐‐‐ Original Message ‐‐‐

On Thursday, September 23rd, 2021 at 1:49 PM, Brandon Williams 
 wrote:

> I think this makes a lot of sense, aligning with the driver. Thanks
>
> for being willing to take up this work!
>
> On Thu, Sep 23, 2021 at 2:37 PM bhouse99
>
> bhous...@protonmail.com.invalid wrote:
>
> > Wiki: 
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH
> >
> > Jira: https://issues.apache.org/jira/browse/CASSANDRA-16456
> >
> > Hi everyone, I’d like to propose the above CEP for adoption by the 
> > community.
> >
> > CQLSH is an important and useful tool but unfortunately lacks the same 
> > support of different authentication methods that can be found in Cassandra 
> > drivers.
> >
> > I propose extending configuration options to bring it more in line with the 
> > Python Cassandra driver. The allows anyone using third party provided 
> > plugins like LDAP, Kerberos, and Sigv4 to be able to continue to use CQSLH.
> >
> > I am prepared to do the principle work to make this change, and how that 
> > the community will adopt the CEP after this discussion.
> >
> > Thanks!
> >
> > Brian

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [DISCUSS] CEP-15: General Purpose Transactions

[Jonathan Ellis]

 The obstacle for me is you've provided a protocol but not a fully fleshed
out architecture, so it's hard to fill in some of the blanks.  But it looks
to me like optimistic concurrency control for interactive transactions
applied to Accord would leave you in a LWT-like situation under fairly
light contention where nobody actually makes progress due to retries.

To make sure we're talking about the same thing, as Henrik pointed out,
interactive transactions mean multiple round trips from the client within a
transaction.  For example, here

is a simple implementation of the TPC-C New Order transaction.  The high
level logic (via
)
is,

   1. Get records describing a warehouse, customer, & district
   2. Update the district
   3. Increment next available order number
   4. Insert record into Order and New-Order tables
   5. For 5-15 items, get Item record, get/update Stock record
   6. Insert Order-Line Record

As you can see, this requires a lot of client-side logic mixed in with the
actual SQL commands.


On Thu, Sep 30, 2021 at 2:30 AM bened...@apache.org 
wrote:

> Essentially this, although I think in practice we will need to track each
> partition’s timestamp separately (or optionally for reduced conflicts, each
> row or datum’s), and make them all part of the conditional application of
> the transaction - at least for strict-serializability.
>
> The alternative is to insert read/write intents for the transaction during
> each step, and to confirm they are still valid on commit, but this approach
> would require a WAN round-trip for each step in the interactive
> transaction, whereas the timestamp-validating approach can use a LAN
> round-trip for each step besides the final one, and is also much simpler to
> implement.
>
>
> From: Blake Eggleston 
> Date: Thursday, 30 September 2021 at 05:47
> To: dev@cassandra.apache.org 
> Subject: Re: [DISCUSS] CEP-15: General Purpose Transactions
> You could establish a lower timestamp bound and buffer transaction state
> on the coordinator, then make the commit an operation that only applies if
> all partitions involved haven’t been changed by a more recent timestamp.
> You could also implement mvcc either in the storage layer or for some
> period of time by buffering commits on each replica before applying.
>
> > On Sep 29, 2021, at 6:18 PM, Jonathan Ellis  wrote:
> >
> > How are interactive transactions possible with Accord?
> >
> >
> >
> > On Tue, Sep 21, 2021 at 11:56 PM bened...@apache.org <
> bened...@apache.org>
> > wrote:
> >
> >> Could you explain why you believe this trade-off is necessary? We can
> >> support full SQL just fine with Accord, and I hope that we eventually
> do so.
> >>
> >> This domain is incredibly complex, so it is easy to reach wrong
> >> conclusions. I would invite you again to propose a system for discussion
> >> that you think offers something Accord is unable to, and that you
> consider
> >> desirable, and we can work from there.
> >>
> >> To pre-empt some possible discussions, I am not aware of anything we
> >> cannot do with Accord that we could do with either Calvin or Spanner.
> >> Interactive transactions are possible on top of Accord, as are
> transactions
> >> with an unknown read/write set. In each case the only cost is that they
> >> would use optimistic concurrency control, which is no worse the spanner
> >> derivatives anyway (which I have to assume is your benchmark in this
> >> regard). I do not expect to deliver either functionality initially, but
> >> Accord takes us most of the way there for both.
> >>
> >>
> >> From: Jonathan Ellis 
> >> Date: Wednesday, 22 September 2021 at 05:36
> >> To: dev 
> >> Subject: Re: [DISCUSS] CEP-15: General Purpose Transactions
> >> Right, I'm looking for exactly a discussion on the high level goals.
> >> Instead of saying "here's the goals and we ruled out X because Y" we
> should
> >> start with a discussion around, "Approach A allows X and W, approach B
> >> allows Y and Z" and decide together what the goals should be and and
> what
> >> we are willing to trade to get those goals, e.g., are we willing to
> give up
> >> global strict serializability to get the ability to support full SQL.
> Both
> >> of these are nice to have!
> >>
> >> On Tue, Sep 21, 2021 at 9:52 PM bened...@apache.org <
> bened...@apache.org>
> >> wrote:
> >>
> >>> Hi Jonathan,
> >>>
> >>> These other systems are incompatible with the goals of the CEP. I do
> >>> discuss them (besides 2PC) in both the whitepaper and the CEP, and will
> >>> summarise that discussion below. A true and accurate comparison of
> these
> >>> other systems is essentially intractable, as there are complex
> subtleties
> >>> to each flavour, and those who are interested would be better served by
> >>> performing their own research.
> >>>
> >>> I think it is more productive t

Re: [DISCUSS] CEP-16: Auth Plugin Support for CQLSH

[Dinesh Joshi]

Hi Brian,

Thanks - the CEP looks good to me. We can move to vote. Could you please 
initiate it?

Dinesh

> On Sep 30, 2021, at 4:27 PM, bhouse99  wrote:
> 
> Hi everyone.  Are there any other thoughts or comments regarding the CEP-16: 
> Auth Plugin Support for CQLSH proposal?
> 
> - 
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH
> 
> If not I'd like to send out a request for a vote.
> 
> Thanks!
> 
> Sent with ProtonMail Secure Email.
> 
> ‐‐‐ Original Message ‐‐‐
> 
> On Thursday, September 23rd, 2021 at 1:49 PM, Brandon Williams 
>  wrote:
> 
>> I think this makes a lot of sense, aligning with the driver. Thanks
>> 
>> for being willing to take up this work!
>> 
>> On Thu, Sep 23, 2021 at 2:37 PM bhouse99
>> 
>> bhous...@protonmail.com.invalid wrote:
>> 
>>> Wiki: 
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-16%3A+Auth+Plugin+Support+for+CQLSH
>>> 
>>> Jira: https://issues.apache.org/jira/browse/CASSANDRA-16456
>>> 
>>> Hi everyone, I’d like to propose the above CEP for adoption by the 
>>> community.
>>> 
>>> CQLSH is an important and useful tool but unfortunately lacks the same 
>>> support of different authentication methods that can be found in Cassandra 
>>> drivers.
>>> 
>>> I propose extending configuration options to bring it more in line with the 
>>> Python Cassandra driver. The allows anyone using third party provided 
>>> plugins like LDAP, Kerberos, and Sigv4 to be able to continue to use CQSLH.
>>> 
>>> I am prepared to do the principle work to make this change, and how that 
>>> the community will adopt the CEP after this discussion.
>>> 
>>> Thanks!
>>> 
>>> Brian
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org