Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
Sorry Jonathan, didn't see this reply earlier today. That would be common behaviour for many MVCC databases, including MongoDB, MySQL Galera Cluster, PostgreSQL... https://www.postgresql.org/docs/9.5/transaction-iso.html *"Applications using this level must be prepared to retry transactions due

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Alex Miller
On Wed, Oct 13, 2021 at 3:52 AM Henrik Ingo wrote: > Aren't you actually pointing out a limitation in any "single shot" > transactional algorithm? Including Accord itself, without any interactive > part? > > What you are saying is that an Accord transaction is limited by the need > for both the cl

Re: [DISCUSS] Cleaning up docs, completing CASSANDRA-16763

2021-10-13 Thread Ekaterina Dimitrova
Hey Stefan, Thank you for your support and spending the time to think about this. If I understand you correctly, you suggest reopening the old tickets to finish the docs, I have two concerns: - we might work a few times on one and the same doc until we bring it to the latest state instead of gett

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread bened...@apache.org
So, if we track per-key timestamps we are able to perform stale reads to assemble our transaction, and validate them on commit. This likely leads to much faster transactions than any other approach, as the interactive part all remains local. If we instead perform Accord operations for every rea

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread Henrik Ingo
On Wed, Oct 13, 2021 at 2:54 PM bened...@apache.org wrote: > I think this is a blurring of lines of systems however. I _think_ the > point Alex is making (correct me if I’m wrong) is that the transaction > system will need to track the transaction timestamps that were witnessed by > each read for

Re: [IDEA] Read committed transaction with Accord

2021-10-13 Thread Henrik Ingo
On Wed, Oct 13, 2021 at 3:38 PM bened...@apache.org wrote: > It may have been lost in the back and forth (there’s been a lot of > emails), but I outlined an approach for READ COMMITTED (and SERIALIZABLE) > read isolation that does not require a WAN trip. Yes, thank you. I knew I had read it but

Re: [IDEA] Read committed transaction with Accord

2021-10-13 Thread bened...@apache.org
> It seems like potentially every statement now needs to go through the Accord > consensus protocol, and this could become expensive, where my goal was to design the simplest and most lightweight example thinkable. BUT for read-only Accord transactions, where I specifically also don't care about s

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread bened...@apache.org
> In the context of Cassandra, I had actually assumed the Accord timestamp will > be used as the cell timestamp for each value? Isn't something like this > needed for compaction to work correctly too? Yes, though we are likely to apply some kind of compression to the timestamp, as global timest

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread bened...@apache.org
I just realised my email client hid a lot of your email, so I now realise I must have misunderstood your statement. I realise now you must have meant per-statement snapshot isolation. However, I believe that MVCC is an optimisation for such an isolation level, not a requirement – it is possible

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Henrik Ingo
Thank you Alex for your feedback, I greatly value these thoughts and always enjoy learning new details in this space. On Wed, Oct 13, 2021 at 10:07 AM Alex Miller wrote: > These two pieces together seem to imply that your claim is that Read > Committed may read whatever the most recently committ

Re: [DISCUSS] CEP-15: General Purpose Transactions

2021-10-13 Thread Henrik Ingo
On Wed, Oct 13, 2021 at 12:25 AM Alex Miller wrote: > I have, purely out of laziness, been engaging on this topic on ASF Slack as > opposed to dev@[1]. Benedict has been overly generous in answering > questions and considering future optimizations there, but it means that I > inadvertently forke

[IDEA] Read committed transaction with Accord

2021-10-13 Thread Henrik Ingo
On Wed, Oct 13, 2021 at 1:26 AM Blake Eggleston wrote: > Hi Henrik, > > I would agree that the local serial experience for valid use cases should > be supported in some form before legacy LWT is replaced by Accord. > > Great! It seems there's a seed of consensus on this point. > Regarding your

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread bened...@apache.org
Jonathan, Your request to separate consensus from execution is about as sensical as asking for this separation in Paxos, or any other distributed consensus protocol. I have made these statements repeatedly, so let me break it down step by step. 1. Accord is an optimal leaderless distributed co

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread bened...@apache.org
Hi Alex, I hugely value and respect your input here, but I think in this case you may be mistaken. Postgres[1] makes explicit that subsequent SELECT statements may see different data, and SQL Server[2] does the same. I believe the Oracle documents you reference do the same, but are more obtuse

Re: [DISCUSS] Cleaning up docs, completing CASSANDRA-16763

2021-10-13 Thread Stefan Miklosovic
Hey, I got an idea how this might be simplified. Every commit in Cassandra repository belongs to a ticket (except ninjas but they are irrelevant here). So if you look over the commits, what about looking at what Jira ticket they belong to (this information is in each commit message) and then go

Re: Tradeoffs for Cassandra transaction management

2021-10-13 Thread Alex Miller
On Tue, Oct 12, 2021 at 3:55 PM Henrik Ingo wrote: > We define READ COMMITTED as "whatever is returned by Cassandra when > executing the query (with QUORUM consistency)". In other words, this > functionality doesn't require any changes to the storage engine or other > fundamental changes to Cassan