Hi Dinesh, Great question! Unfortunately I don’t have a great definition of what “alpha” means in the Cassandra community so its hard for me to answer that directly. However, I am of the opinion that we are not yet at the point of being able to branch trunk — we are finding too many bugs at too quick a pace still and have yet to make enough significant progress on the test plan [1] previously linked. I do think it would be beneficial to cut an official build (maybe after internode messaging settles down) as a preview for the community and to make it easier for folks to run on dev/test hardware. In the Riak community we call these “pre” builds (Riak 2.0.0preX) and they were nothing more than a stable place on trunk released periodically until we reached a point where we branched.
Regarding metrics, the first major step towards that was Benedict’s and others work (thanks al!) to re-organize JIRA. We now have a better set of inputs to automatically build reports around release quality metrics, etc. We have yet to take this and turn it into JIRA reports but I am working with Scott Andreas on it — I don’t have a timeframe just yet but I hope soon. If you would like to help please let me know. In the meantime, Scott and I have kept a list which is where the data I used came from. We absolutely need to make this public and the efforts mentioned above will accomplish that. Jordan [1] https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans On Thu, Apr 11, 2019 at 4:21 PM Dinesh Joshi <djo...@apache.org> wrote: > Hey Jordan, > > Thanks for update! Do you have a sense of where we are in terms of > stability and where do we need to be in order to cut an alpha? I also > remember a discussion on measuring release quality[1]. Not sure where we > landed on it. Any idea on how are we doing on that front? > > Thanks, > > Dinesh > > [1] > https://lists.apache.org/thread.html/3a444be1a3097c0c55d15268ccb0fe7aab83ef276b87bf55bf4f3bc2@%3Cdev.cassandra.apache.org%3E > > > On Apr 10, 2019, at 8:25 AM, Jordan West <jorda...@gmail.com> wrote: > > > > In September, the community chose to freeze trunk to begin working on > > Quality and Stability with the goal of releasing the most stable > Cassandra > > major in the project’s history. While lots of work has been ongoing and > > folks could follow along with progress on JIRA I thought it would be > useful > > to cover what has been accomplished so far since I’ve spent a good amount > > of time working with others on various testing projects. > > > > During this time we have made significant progress on improving the > Quality > > and Stability of Cassandra — not only Cassandra 4.0 but also the > Cassandra > > 3.x series and future Cassandra releases. Additionally, testing has > > provided the opportunity for new community members and committers to > > contribute. While not comprehensive the community has found at least 25 > > bugs that can be classified as either Data Loss, Corruption, Incorrect > > Response, Loss of Stability, Loss of Availability, Concurrency Issues, > > Performance Issues, and Lack of Safety. These bugs have been found by a > > variety of methodologies including commonly used ones like unit testing > and > > canary deployments. However, the majority of the bugs have been found or > > confirmed using new methodologies like the ones described in a some > recent > > blog posts [1] [2]. > > > > Additionally, the state of the test suites and test tooling have > improved. > > CASSANDRA-14806 [3] brought some much welcomed improvements to the > circleci > > workflow and made it easier for people to run (d)tests on supported > > platforms (jdk8/11) and the work to get upgrade tests running found > several > > bugs including CASSADNRA-14958 [4]. > > > > While we have made significant progress there is still more to do before > we > > can be truly confident in an Cassandra 4.0 release. Some ongoing and > > outstanding work includes: > > > > * Improving the state of the cqlsh tests [5] > > * There is ongoing discussion on the new MessagingService [6] which will > > require significant review and testing > > * Additional upgrade testing for Cassandra 4.0 including additional > support > > for upgrade testing using in-jvm dtests [7] > > * Work to increase coverage of important areas and new features in > > Cassandra 4.0 [8] > > > > While the list above may seem short, the last item contains a long list > of > > important areas the community has previously discussed adding coverage > to. > > If you are looking for areas to contribute this is a great starting > point. > > If there is a name down on an area you are interested in I would > encourage > > you to reach out to them to discuss how you can help further increase the > > community’s confidence in the Quality and Stability of Cassandra. > > > > Below is an in-complete list of many of the severe bugs found during this > > part of the release cycle. Thanks again to all of the community members > who > > contributed to finding these bugs and improving Cassandra for everyone. > > > > CASSANDRA-15004: Anti-compaction briefly removes sstables from the read > path > > CASSANDRA-14958: Counters fail to increment on 2.X to 3.X mixed version > > clusters > > CASSANDRA-14936: Anticompaction should throw exceptions on errors, not > just > > log them > > CASSANDRA-14672: After deleting data in 3.11.3, reads fail: "open marker > > and close marker have different deletion times" > > CASSANDRA-14912: LegacyLayout errors on collection tombstones from > dropped > > columns > > CASSANDRA-14843: Drop/add column name with different Kind can result in > > corruption > > CASSANDRA-14568: CorruptSSTableExceptions in 3.0.17.1 (CASSANDRA-14568 > v2) > > Static collection deletions are corrupted in 3.0 <-> 2.{1,2} messages > > CASSANDRA-14749: Collection Deletions for Dropped Columns in 2.1/3.0 > > mixed-mode can delete rows > > CASSANDRA-14568: Static collection deletions are corrupted in 3.0 -> > > 2.{1,2} messages > > CASSANDRA-14861: Inaccurate sstable min/max metadata can cause data loss > > CASSANDRA-14823: Legacy sstables with range tombstones spanning multiple > > index blocks create invalid bound sequences on 3.0+ (#1193) > > CASSANDRA-14873: Missing rows when reading 2.1 SSTables in 3.0 > > CASSANDRA-14838: Dropped columns can cause reverse sstable iteration to > > return prematurely > > CASSANDRA-14803: Rows that cross index block boundaries can cause > > incomplete reverse reads in some cases. > > CASSANDRA-14766: DESC order reads can fail to return the last Unfiltered > in > > the partition (#1170) > > CASSANDRA-14991: SSL Cert Hot Reloading should defensively check for > sanity > > of the new keystore/truststore before loading it > > CASSANDRA-14794: Avoid calling iter.next() in a loop when notifying > > indexers about range tombstones > > CASSANDRA-14780: Avoid creating empty compaction tasks after truncate > > CASSANDRA-14657: Handle failures in upgradesstables/cleanup/relocatee > > CASSANDRA-14638: Column result order can change in 'SELECT *' results > when > > upgrading from 2.1 to 3.0 causing response corruption for queries using > > prepared statements when static columns are used > > CASSANDRA-14919: Regression in paging queries in mixed version clusters > > CASSANDRA-14554: LifecycleTransaction encounters > > ConcurrentModificationException when used in multi-threaded context > > CASSANDRA-14935: PendingAntiCompaction should be more judicious in the > > compactions it cancels > > CASSANDRA-14894: RangeTombstoneList doesn't properly clean up mergeable > or > > superseded rts in some cases > > CASSANDRA-14824: Expand range tombstone validation checks to multiple > > interim request stages > > CASSANDRA-14763: Fail incremental repair prepare phase if it encounters > > sstables from un-finalized sessions > > CASSANDRA-14920: Some comparisons used for verifying paging queries in > > dtests only test the column names and not values > > > > Jordan > > > > [1] > > > http://cassandra.apache.org/blog/2018/08/21/testing_apache_cassandra.html > > [2] > > > http://cassandra.apache.org/blog/2018/10/17/finding_bugs_with_property_based_testing.html > > [3] https://issues.apache.org/jira/browse/CASSANDRA-14806 > > [4] https://issues.apache.org/jira/browse/CASSANDRA-14958 > > [5] https://issues.apache.org/jira/browse/CASSANDRA-14951 > > [6] https://issues.apache.org/jira/browse/CASSANDRA-15066 > > [7] https://issues.apache.org/jira/browse/CASSANDRA-15078 > > [8] > > > https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
