Re: [DISCUSSION] Workshop idea
+1. Great idea. A virtual meetup, Introduction to Cassandra 4.0, will help users try and test 4.0. ~Charles On Tue, Jul 28, 2020 at 9:26 AM Ekaterina Dimitrova wrote: > > Hi Nate, all, > I was thinking about a 1 hour online talk on 4.0, new features, testing, etc. > I guess it can be also recorded and distributed for those who missed the live > session. Slides also could be added to slide share. > As you mentioned demos, that sounds good to me if people have the time for > demos to be recorded and distributed. > > Best regards, > Ekaterina > > > On 27 Jul 2020, at 21:29, Nate McCall wrote: > > > > This is a really interesting idea, particularly given that at this point in > > our release cycle previously, there would be demos/roadshows happening at > > meetups etc. that we just cant do these days in most places now. > > > > What is your thinking on format? > > > > +1 in general though - that goes for anybody that wants to do something > > like this. > > > > > > > > On Tue, Jul 28, 2020 at 12:21 PM Ekaterina Dimitrova > > wrote: > > > >> Hello everyone, > >> I hope everyone is doing well in these weird pandemic times. > >> > >> I am super excited to see people already testing 4.0 Beta and coming back > >> with feedback. To me the user experience is one of the most important > >> factors for a successful release. > >> While talking about ApacheCon™ last week, an idea was born. How does the > >> community feel about having a workshop on Cassandra 4.0 in August? > >> Why do I find this idea great? > >> 1) Let’s keep the momentum going and try to get the attention of even more > >> users to start testing. > >> 2) ApacheCon™ is only at the end of September, so we can also use the > >> opportunity to heat up before the conference. > >> There is no plan or anything at this point. Just an idea was born and I am > >> excited to share it with all of you and get some > >> feedback/suggestions/criticism. > >> How does the rest of you feel about it? > >> Looking forward to hearing from you. > >> Best regards, > >> Ekaterina > >> - > >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > >> For additional commands, e-mail: dev-h...@cassandra.apache.org > >> > >> > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Cassandra CI Status – 2020-08-08
This is awesome! Thanks Mick for all the great work. ~Charles On Mon, Aug 10, 2020 at 11:41 AM Ekaterina Dimitrova wrote: > > Thank you Mick for all the work you are doing for the C* CI and not only! > I am especially excited about the addition of the upgrade tests. > It’s also great that things got documented. > > Best regards, > Ekaterina > > On Sat, 8 Aug 2020 at 6:41, Mick Semb Wever wrote: > > > The following is a summary of changes, status, and suggestions to our > > community CI, ci-cassandra.apache.org > > Please reply with questions, as well as any input on CircleCI status anyone > > has to offer. > > > > This post will touch on… > > * Upgrade Tests > > * Build Times and Improvements > > * Pre Commit Builds > > * Stand-alone Pipeline Runs > > * Standardising our CI Build Scripts > > * JDK11 > > * Nightly Build Artefacts > > * CI Documentation > > * 4.0 QA Status > > > > > > ** Upgrade Tests > > > > Both in-jvm and normal upgrade tests have been added to > > ci-cassandra.apache.org > > > > Those in-jvm upgrade tests have been included in the pipeline builds. The > > normal upgrade tests currently remain stand-alone. Trunk’s version is found > > here https://ci-cassandra.apache.org/job/Cassandra-trunk-dtest-upgrade/ > > > > > > ** Build Times and Improvements > > > > DTests have been parallelised. By default DTest jobs are divided into 64 > > splits now, with the exception of the Large DTest jobs which due to having > > far fewer tests have only 8 splits. > > > > This brings dtest runs from ~12 hours down to ~45 minutes. It brings whole > > pipeline builds from ~14 hours down to ~2.5 hours. Some patch (devbranch) > > builds have completed in 90 minutes. > > > > For more speed we can split more, but ci-cassandra currently has 36 agents > > (72 executors) and is now often saturated and build queues large. We can > > also look into Unit Tests which are only ever using one runner on both > > ci-cassandra and circleci. A problem here is that using more runners breaks > > some of the unit tests. Another possible improvement is to avoid the > > compiling in all the test stages, by re-using the built artefacts in the > > beginning of each pipeline run. > > > > > > ** Pre Commit Builds > > > > ci-cassandra.apache.org is less frequently used for pre-commit builds, for > > reasons of resource limits and access reserved to committers. More > > information on this can be found in > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=153815764 > > > > Some trusted contributors have been given Jenkins API tokens, along with > > using > > the script found here > > https://the-asf.slack.com/archives/C0162JU2CKY/p1595703740008400 > > > > These tokens do rotate, and how/when still remains a little unclear. > > Committers can generate tokens via their Jenkins profile pages. There’s > > also investigation as to whether we can create jenkins accounts with build > > permissions for trusted contributors. Again with the resources available, > > especially in contrast to additional testing that will probably appear in > > the 4.0 beta testing phase, we are restricted to what is feasible here. > > > > Notifications for all devbranch pipeline builds now go to > > #cassandra-builds-patches. These report the build url, the commit SHA and > > message, and the patch repository. > > > > > > ** Stand-alone Pipeline Runs > > > > It has been raised a number of times that it would be great if users > > (companies) could run the complete pipeline on their own resources/cloud, > > from a single command line, including the setup and teardown of the jenkins > > platform. This would be a big win for the community with standardised > > testing and test reports to share, and helping to test all the possible > > configuration combinations possible. There is some work involved to do this > > but we appear to be moving in that direction anyway. For it to happen the > > all stage jobs inside the pipeline need to be moved, from being generated > > in the dsl script, to being defined in the in-tree Jenkinsfile. > > > > > > ** Standardising our CI Build Scripts > > > > Today we have a lot of duplication of build scripts. Those in > > cassandra-builds/build-scripts/ and those embedded into each of the > > circleci config files in-tree. > > > > I would like to suggest we move the build-scripts in-tree, and start > > migrating circleci to re-use the same build scripts. There are differences > > from how test lists are split (round-robin `split` to circleci timings > > based splitting) to how parallelisation works (circle’s containers vs the > > jenkins matrix plugin), but I suspect by focusing on the easy stuff there’s > > a lot that can be standardised. > > > > > > ** JDK11 > > > > JDK11 builds have been contributed, thanks to Shylaja. Trunk’s pipeline now > > builds both JDK 8 and 11 artefacts. > > > > Adding JDK11 test runs hit a hurdle with how the JDK labels are named and > > our tests are no
Re: [DISCUSS] CEP-7 Storage Attached Index
Patrick, 11AM PST is a bad time for the people in the APAC timezone. Can we move it to 7 or 8AM PST in the morning to accommodate their needs ? ~Charles On Fri, Aug 28, 2020 at 4:37 PM Patrick McFadin wrote: > > Meeting scheduled. > https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting > > Tuesday September 1st, 11AM PST. I added a basic bullet for the agenda but > if there is more, edit away. > > Patrick > > On Thu, Aug 27, 2020 at 11:31 AM Jasonstack Zhao Yang < > jasonstack.z...@gmail.com> wrote: > > > +1 > > > > On Thu, 27 Aug 2020 at 04:52, Ekaterina Dimitrova > > wrote: > > > > > +1 > > > > > > On Wed, 26 Aug 2020 at 16:48, Caleb Rackliffe > > > wrote: > > > > > > > +1 > > > > > > > > > > > > > > > > On Wed, Aug 26, 2020, 3:45 PM Patrick McFadin > > > wrote: > > > > > > > > > > > > > > > > > This is related to the discussion Jordan and I had about the > > > contributor > > > > > > > > > Zoom call. Instead of open mic for any issue, call it based on a > > > > discussion > > > > > > > > > thread or threads for higher bandwidth discussion. > > > > > > > > > > > > > > > > > > I would be happy to schedule on for next week to specifically discuss > > > > > > > > > CEP-7. I can attach the recorded call to the CEP after. > > > > > > > > > > > > > > > > > > +1 or -1? > > > > > > > > > > > > > > > > > > Patrick > > > > > > > > > > > > > > > > > > On Tue, Aug 25, 2020 at 7:03 AM Joshua McKenzie < > > jmcken...@apache.org> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Does community plan to open another discussion or CEP on > > > > > > > > > modularization? > > > > > > > > > > > > > > > > > > > > We probably should have a discussion on the ML or monthly contrib > > > call > > > > > > > > > > about it first to see how aligned the interested contributors are. > > > > Could > > > > > > > > > do > > > > > > > > > > that through CEP as well but CEP's (at least thus far sans k8s > > > > operator) > > > > > > > > > > tend to start with a strong, deeply thought out point of view being > > > > > > > > > > expressed. > > > > > > > > > > > > > > > > > > > > On Tue, Aug 25, 2020 at 3:26 AM Jasonstack Zhao Yang < > > > > > > > > > > jasonstack.z...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > >>> SASI's performance, specifically the search in the B+ tree > > > > > > > > > component, > > > > > > > > > > > >>> depends a lot on the component file's header being available > > in > > > > the > > > > > > > > > > > >>> pagecache. SASI benefits from (needs) nodes with lots of RAM. > > > Is > > > > > > > > > SAI > > > > > > > > > > > bound > > > > > > > > > > > >>> to this same or similar limitation? > > > > > > > > > > > > > > > > > > > > > > SAI also benefits from larger memory because SAI puts block info > > on > > > > > > > > > heap > > > > > > > > > > > for searching on-disk components and having cross-index files on > > > page > > > > > > > > > > cache > > > > > > > > > > > improves read performance of different indexes on the same table. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >>> Flushing of SASI can be CPU+IO intensive, to the point of > > > > > > > > > saturation, > > > > > > > > > > > >>> pauses, and crashes on the node. SSDs are a must, along with > > a > > > > bit > > > > > > > > > of > > > > > > > > > > > >>> tuning, just to avoid bringing down your cluster. Beyond > > > reducing > > > > > > > > > > space > > > > > > > > > > > >>> requirements, does SAI improve on these things? Like SASI how > > > > does > > > > > > > > > > SAI, > > > > > > > > > > > in > > > > > > > > > > > >>> its own way, change/narrow the recommendations on node > > hardware > > > > > > > > > > specs? > > > > > > > > > > > > > > > > > > > > > > SAI won't crash the node during compaction and requires less > > > CPU/IO. > > > > > > > > > > > > > > > > > > > > > > * SAI defines global memory limit for compaction instead of > > > per-index > > > > > > > > > > > memory limit used by SASI. > > > > > > > > > > > For example, compactions are running on 10 tables and each has > > 10 > > > > > > > > > > > indexes. SAI will cap the > > > > > > > > > > > memory usage with global limit while SASI may use up to 100 * > > > > > > > > > per-index > > > > > > > > > > > limit. > > > > > > > > > > > > > > > > > > > > > > * After flushing in-memory segments to disk, SAI won't merge > > > on-disk > > > > > > > > > > > segments while SASI > > > > > > > > > > > attempts to merge them at the end. > > > > > > > > > > > > > > > > > > > > > > There are pros and cons of not merging segments: > > > > > > > > > > > ** Pros: compaction runs faster and requires fewer resources. > > > > > > > > > > > ** Cons: small segments reduce compression ratio. > > > > > > > > > > > > > > > > > > > > > > * SAI on-disk format with row ids compresses better. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >>> I understand the d
Re: [DISCUSS] CEP-7 Storage Attached Index
Thank you, Patrick! On Mon, Aug 31, 2020 at 12:59 PM Patrick McFadin wrote: > > I just moved it to 8AM for this meeting to better accommodate APAC. Please > see the update here: > https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting > > Patrick > > On Mon, Aug 31, 2020 at 10:04 AM Charles Cao wrote: > > > Patrick, > > > > 11AM PST is a bad time for the people in the APAC timezone. Can we > > move it to 7 or 8AM PST in the morning to accommodate their needs ? > > > > ~Charles > > > > On Fri, Aug 28, 2020 at 4:37 PM Patrick McFadin > > wrote: > > > > > > Meeting scheduled. > > > > > https://cwiki.apache.org/confluence/display/CASSANDRA/2020-08-01+Apache+Cassandra+Contributor+Meeting > > > > > > Tuesday September 1st, 11AM PST. I added a basic bullet for the agenda > > but > > > if there is more, edit away. > > > > > > Patrick > > > > > > On Thu, Aug 27, 2020 at 11:31 AM Jasonstack Zhao Yang < > > > jasonstack.z...@gmail.com> wrote: > > > > > > > +1 > > > > > > > > On Thu, 27 Aug 2020 at 04:52, Ekaterina Dimitrova < > > e.dimitr...@gmail.com> > > > > wrote: > > > > > > > > > +1 > > > > > > > > > > On Wed, 26 Aug 2020 at 16:48, Caleb Rackliffe < > > calebrackli...@gmail.com> > > > > > wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Aug 26, 2020, 3:45 PM Patrick McFadin > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > This is related to the discussion Jordan and I had about the > > > > > contributor > > > > > > > > > > > > > Zoom call. Instead of open mic for any issue, call it based on a > > > > > > discussion > > > > > > > > > > > > > thread or threads for higher bandwidth discussion. > > > > > > > > > > > > > > > > > > > > > > > > > > I would be happy to schedule on for next week to specifically > > discuss > > > > > > > > > > > > > CEP-7. I can attach the recorded call to the CEP after. > > > > > > > > > > > > > > > > > > > > > > > > > > +1 or -1? > > > > > > > > > > > > > > > > > > > > > > > > > > Patrick > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Aug 25, 2020 at 7:03 AM Joshua McKenzie < > > > > jmcken...@apache.org> > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Does community plan to open another discussion or CEP on > > > > > > > > > > > > > modularization? > > > > > > > > > > > > > > > > > > > > > > > > > > > > We probably should have a discussion on the ML or monthly > > contrib > > > > > call > > > > > > > > > > > > > > about it first to see how aligned the interested contributors > > are. > > > > > > Could > > > > > > > > > > > > > do > > > > > > > > > > > > > > that through CEP as well but CEP's (at least thus far sans k8s > > > > > > operator) > > > > > > > > > > > > > > tend to start with a strong, deeply thought out point of view > > being > > > > > > > > > > > > > > expressed. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Aug 25, 2020 at 3:26 AM Jasonstack Zhao Yang < > > > > > > > > > > > > > > jasonstack.z...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > &g
Re: Welcome Yifan Cai as Cassandra committer
Congrats YiFan ! ~Charles > On Dec 21, 2020, at 09:27, sankalp kohli wrote: > > Congratulations Yifan. > >> On Mon, Dec 21, 2020 at 9:10 AM Benjamin Lerer >> wrote: >> >> The PMC's members are pleased to announce that Yifan Cai has accepted the >> invitation to become committer last Friday. >> >> Thanks a lot, Yifan, for everything you have done! >> >> Congratulations and welcome >> >> The Apache Cassandra PMC members >> - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Welcome Caleb Rackliffe as Cassandra committer
Congrats Caleb! Well deserved :) ~Charles > On May 14, 2021, at 07:30, Yifan Cai wrote: > > Congrats Caleb! > >> On May 14, 2021, at 6:56 AM, Joshua McKenzie wrote: >> >> Congrats Caleb! >> On Fri, May 14, 2021 at 9:10 AM Brandon Williams wrote: >>> >>> Congrats Caleb! Well deserved. >>> On Fri, May 14, 2021, 8:03 AM Mick Semb Wever wrote: The PMC members are pleased to announce that Caleb Rackliffe has accepted the invitation to become committer. Thanks heaps Caleb for helping make Cassandra awesome! Congratulations and welcome, The Apache Cassandra PMC members >>> > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: Thanks to Nate for his service as PMC Chair
Thanks Nate and Congrats Mike for the new role! ~Charles > On Jul 11, 2022, at 08:03, Mick Semb Wever wrote: > > > Thank you Nate, and Paulo and everyone! 🙏 > > >> On Mon, 11 Jul 2022 at 15:57, Ekaterina Dimitrova >> wrote: >> Thanks Nate and all the best with your new role Mick! >> >>> On Mon, 11 Jul 2022 at 9:01, Jeremiah Jordan wrote: >>> Thanks Nate! And welcome to the role Mick! >>> > On Jul 11, 2022, at 7:54 AM, Paulo Motta wrote: > >>> Hi, I wanted to announce on behalf of the Apache Cassandra Project Management Committee (PMC) that Nate McCall (zznate) has stepped down from the PMC chair role. Thank you Nate for all the work you did as the PMC chair! The Apache Cassandra PMC has nominated Mick Semb Wever (mck) as the new PMC chair. Congratulations and good luck on the new role Mick! The chair is an administrative position that interfaces with the Apache Software Foundation Board, by submitting regular reports about project status and health. Read more about the PMC chair role on Apache projects: - https://www.apache.org/foundation/how-it-works.html#pmc - https://www.apache.org/foundation/how-it-works.html#pmc-chair - https://www.apache.org/foundation/faq.html#why-are-PMC-chairs-officers The PMC as a whole is the entity that oversees and leads the project and any PMC member can be approached as a representative of the committee. A list of Apache Cassandra PMC members can be found on: https://cassandra.apache.org/_/community.html Kind regards, Paulo
