I welcome this development - honestly, possibly long overdue. Would it be possible to propose additional committers for a wider representation?
Thank you, Renat -----Original Message----- From: larry mccay <[email protected]> Sent: Monday, October 24, 2022 4:41 PM To: [email protected] Subject: [EXTERNAL] Re: Proposal to Revive Apache Livy Community Again, I have no objections to this and will be happy to include you. Thanks for calling out! On Mon, Oct 24, 2022 at 5:38 PM Marco Gaido <[email protected]> wrote: > Hi all, > > I am also busy with different projects at the moment, as such I am > more than happy if ai can help somehow, e.g. reviewing code in parts > that I know, but I cannot allocate much bandwidth to it. > > Thanks, > Marco > > Il lun 24 ott 2022, 22:07 larry mccay <[email protected]> ha scritto: > > > Personally, I would welcome it. > > Any notes of guidance and support as we ramp up the team and make > > the > first > > few releases would be invaluable. > > > > > > On Mon, Oct 24, 2022 at 3:49 PM Alex Bozarth <[email protected]> > wrote: > > > > > I would love to stay on as a PPMC, but if my lack of availability > > concerns > > > the new PPMC then I'm willing to be emeritus. > > > > > > > > > Alex Bozarth > > > Jupyter Architect, IBM CODAIT > > > GitHub: ajbozarth > > > > > > On 10/24/22, 2:46 PM, "larry mccay" <[email protected]> wrote: > > > > > > Oh, very good, Alex! > > > Be sure to let us know if you want to remain on the project in > > > any capacity. > > > > > > > > > On Mon, Oct 24, 2022 at 3:18 PM Alex Bozarth > > > <[email protected]> > > > wrote: > > > > > > > As the only Livy PPMC still responding to the Mentors on the > > private > > > list, > > > > I have updated the Livy Podling report for November with the > status > > > of the > > > > project and a request to the IPMC to review this proposal > > > since there are > > > > not enough active Livy PPMC members to reach a quorum to > > > pass the proposal. > > > > > > > > As a current Livy PPMC I strongly support this proposal for > > > revitalization > > > > as I do not have enough bandwidth to dedicate to Livy. > > > > > > > > > > > > Alex Bozarth > > > > Jupyter Architect, IBM CODAIT > > > > GitHub: ajbozarth > > > > > > > > On 10/24/22, 1:41 PM, "larry mccay" <[email protected]> wrote: > > > > > > > > Gentle reminder that we need to determine next steps here. > > > > We have an updated proposal on this thread. > > > > Do we need a VOTE or can we move forward directly to > adjusting > > > the > > > > members, > > > > etc? > > > > > > > > Thanks! > > > > > > > > --larry > > > > > > > > On Thu, Oct 20, 2022 at 3:26 PM larry mccay < > [email protected] > > > > > > wrote: > > > > > > > > > @Justin Mclean <[email protected]> - any insights on > next > > > steps > > > > here? > > > > > > > > > > > > > > > On Tue, Oct 18, 2022 at 5:44 PM larry mccay < > > [email protected] > > > > > > > > wrote: > > > > > > > > > >> Very good, here is the latest revision with updated > Mentors. > > > > >> Sunil and I have been added to the IPMC as well. > > > > >> Welcome Madhawa and thanks for stepping up as a Mentor for > > > Livy! > > > > >> > > > > >> Abstract > > > > >> > > > > >> Livy is a web service that exposes a REST interface for > > > managing > > > > long > > > > >> running Apache Spark contexts in your cluster. With Livy, > > new > > > > applications > > > > >> can be built on top of Apache Spark that require fine > > grained > > > > interaction > > > > >> with many Spark contexts [1]. > > > > >> > > > > >> While this project has been well regarded and used in many > > > contexts > > > > as > > > > >> the defacto standard API to Spark environments, it has > been > > > > incubating for > > > > >> over 5 years without graduation to a TLP and it has become > > > > difficult to > > > > >> impossible for fixes and improvements to be contributed as > > the > > > > current > > > > >> community seems to have moved on. > > > > >> > > > > >> There has been discussion regarding retirement of this > > > podling where > > > > >> there seems to be some increasing interest in joining and > > > reviving > > > > the > > > > >> community [2]. > > > > >> > > > > >> The intent of this proposal is to avoid retiring a well > > > regarded, > > > > >> actively used and rather mature project by reviving the > PPMC > > > and > > > > community > > > > >> with new folks that have a vested interest in the project > > and > > > > health of the > > > > >> community. > > > > >> Proposal > > > > >> > > > > >> We propose to revive the PPMC with a set of contributors > and > > > > maintainers > > > > >> as mentors, PPMC members and committers. > > > > >> > > > > >> The retirement DISCUSS thread [2] has shown a growing > > > interest in > > > > >> providing new committers and bringing improvements and > fixes > > > from > > > > >> organization’s internally maintained forks back to a > revived > > > > community. > > > > >> > > > > >> General Approach to Revival: > > > > >> > > > > >> - > > > > >> > > > > >> Add new Mentors > > > > >> - > > > > >> > > > > >> Larry McCay, [email protected] , Cloudera > > > > >> - > > > > >> > > > > >> Sunil Govindan, [email protected], Cloudera > > > > >> - > > > > >> > > > > >> Jean-Baptiste Onofré, [email protected], Talend > > > > >> - > > > > >> > > > > >> Madhawa Gunasekara, [email protected], > Independent > > > > >> > > > > >> > > > > >> > > > > >> - > > > > >> > > > > >> Add new Committers/PPMC > > > > >> - > > > > >> > > > > >> Larry McCay, [email protected], Cloudera > > > > >> - > > > > >> > > > > >> Vinod Kumar Vavilapalli, [email protected], > > Cloudera > > > > >> - > > > > >> > > > > >> Imran Rashid - [email protected], Cloudera > > > > >> - > > > > >> > > > > >> Gyorgy Gal, ggal ,[email protected], Cloudera > > > > >> - > > > > >> > > > > >> Wing Yew Poon, [email protected], Cloudera > > > > >> - > > > > >> > > > > >> Xilang Yan, [email protected], Shopee > > > > >> - > > > > >> > > > > >> Jianzhen Wu, [email protected], Shopee > > > > >> - > > > > >> > > > > >> Nagella Jagadeewara Rao, [email protected], Visa > > > > >> - > > > > >> > > > > >> Pralab Kumar, [email protected], Visa > > > > >> - > > > > >> > > > > >> Prasad Shrikant, [email protected], Visa > > > > >> - > > > > >> > > > > >> Brahma Reddy Battula, [email protected], Visa > > > > >> > > > > >> > > > > >> > > > > >> - > > > > >> > > > > >> Invite existing PPMC members to opt-in or otherwise go > > > emeritus > > > > >> - > > > > >> > > > > >> Jean-Baptiste Onofré, [email protected], Talend > > > (opted-in > > > > via > > > > >> Retirement DISCUSS thread [2]) > > > > >> > > > > >> > > > > >> > > > > >> - > > > > >> > > > > >> Invite existing Committers to opt-out or otherwise > > continue > > > > >> > > > > >> > > > > >> > > > > >> - > > > > >> > > > > >> Establish Roadmap via follow up DISCUSS thread > > > > >> - > > > > >> > > > > >> Known Improvements from Forks which will need > > proposals > > > and > > > > >> discussion: > > > > >> - > > > > >> > > > > >> Adding HA for Livy > > > > >> - > > > > >> > > > > >> Updating security capabilities (eg. kerberos for > > > jdbc, > > > > fixing > > > > >> bugs in encryption) > > > > >> - > > > > >> > > > > >> Expanding the support for kubernetes > > > > >> - > > > > >> > > > > >> Responding to CVEs in dependencies (eg. log4j, > > > thrift) > > > > >> - > > > > >> > > > > >> Livy rest cluster - IS THIS SAME AS HA for Livy > > > ABOVE? > > > > >> - > > > > >> > > > > >> Support multi Spark versions > > > > >> - > > > > >> > > > > >> Implemented a metrics system for Livy > > > > >> - > > > > >> > > > > >> Support customize batch/interactive session > > > lifecycle event > > > > >> handler, default log event with log4j, very > helpful > > > for > > > > troubleshooting > > > > >> - > > > > >> > > > > >> Optimize log to track which session id the log > > > message came > > > > >> from, also very helpful for troubleshooting > > > > >> - > > > > >> > > > > >> Support customize Spark config optimization > rules, > > > can be > > > > used > > > > >> to optimize config for users’ job > > > > >> - > > > > >> > > > > >> A set of command line tool which can be used to > > > replace > > > > Spark’s > > > > >> spark-submit, pyspark, spark-sql but actually > > submit > > > > application in Livy > > > > >> - > > > > >> > > > > >> We are planning to implement a JDBC state store, > > and > > > allow > > > > >> multi Livy Thrift sessions to share one backend > > Spark > > > > application in the > > > > >> next few months. > > > > >> - > > > > >> > > > > >> These items and others that are brought to community > > > may need > > > > >> consolidation or multiple configurable options and > > will > > > need > > > > to be part of > > > > >> the discussion > > > > >> - > > > > >> > > > > >> One-pager Livy Improvement Proposals (LIP) may > make > > > sense > > > > to > > > > >> drive these discussions and convergence > > > > >> - > > > > >> > > > > >> Feature Branch Strategy for large changes > > > > >> - > > > > >> > > > > >> Large features are hard to review we will need > > to > > > > define a > > > > >> strategy > > > > >> - > > > > >> > > > > >> Determine the Improvements to be delivered across > > first > > > 3 > > > > Releases > > > > >> with Target Release Dates > > > > >> > > > > >> > > > > >> > > > > >> - > > > > >> > > > > >> Ensure CVE and Dependency management hygiene is in > place > > > > >> > > > > >> > > > > >> The above approach will usher the community back to an > > active > > > > status with > > > > >> a Roadmap of 3 or more release plans and security hygiene > in > > > place. > > > > >> Development Practices > > > > >> > > > > >> The Livy project follows a review before commit > philosophy. > > > Every > > > > commit > > > > >> automatically runs through the unit tests and generates > > > coverage > > > > reports > > > > >> presented as a pull request comment. Our experience with > > this > > > > process leads > > > > >> us to believe that it helps ease new contributors into the > > > project. > > > > They > > > > >> get feedback quickly on common mistakes, lowering the > burden > > > on > > > > reviewers. > > > > >> Those same reviewers get to lead by example, showing the > new > > > > contributors > > > > >> that we value feedback within our community even when > > changes > > > are > > > > done by > > > > >> more experienced folks. Taken from the original Apache > Livy > > > > Proposal [1], > > > > >> this should continue to be true. As mentioned, Livy is a > > > mature > > > > project and > > > > >> as such RTC is the most appropriate for continued quality > > and > > > > awareness. > > > > >> > > > > >> > > > > >> > > > > >> 1. > > > > >> > > > > >> Original Apache Livy Proposal > > > > >> > > > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwik > i.apache.org%2Fconfluence%2Fdisplay%2Fincubator%2FLivyProposal&dat > a=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee > 9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535784379%7CUn > known%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haW > wiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=rbccZA%2BUAUHjpEolIiY8ig%2B > 5nRw5abmw9oiwJGkz8TM%3D&reserved=0 > > > > >> 2. > > > > >> > > > > >> Retirement DISCUSS thread > > > > >> > > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fgcstsrhbp91c5mm55htqn1l3djv8m7o0&data=05%7C01%7Crebekbol%40microsoft.com%7C5ead1754350a408bceba08dab6192ee9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638022516535940162%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dgaMvjevmL84UXIvyHZ3OgCDX%2FXs7ZdKZvNXR1UR5A8%3D&reserved=0 > > > > > > > > >> > > > > >> > > > > >> On Tue, Oct 18, 2022 at 5:05 PM Madhawa Gunasekara < > > > > [email protected]> > > > > >> wrote: > > > > >> > > > > >>> Hi Larry, > > > > >>> > > > > >>> I'm an IPMC Member. madhawa30 at gmail dot com is my > > > preferred > > > > email > > > > >>> address. > > > > >>> apache id: madhawa > > > > >>> > > > > >>> Thanks, > > > > >>> Madhawa > > > > >>> > > > > >>> > > > > >>> On Tue, Oct 18, 2022 at 10:05 PM larry mccay < > > > [email protected]> > > > > wrote: > > > > >>> > > > > >>> > Hi Madhawa - > > > > >>> > > > > > >>> > That's awesome! > > > > >>> > Are you already a member of IPMC? > > > > >>> > If not, are you an ASF member? > > > > >>> > If you are an ASF member you can request that you be > > added > > > as an > > > > IPMC > > > > >>> > member. > > > > >>> > > > > > >>> > Can you provide your company affiliation for the > proposal > > > and > > > > preferred > > > > >>> > email? > > > > >>> > > > > > >>> > thanks! > > > > >>> > > > > > >>> > --larry > > > > >>> > > > > > >>> > On Tue, Oct 18, 2022 at 2:18 PM Madhawa Gunasekara < > > > > >>> [email protected]> > > > > >>> > wrote: > > > > >>> > > > > > >>> > > Hi Larry, > > > > >>> > > > > > > >>> > > I'm interested in working with Livy and would like to > > > join as a > > > > >>> Mentor. > > > > >>> > > > > > > >>> > > Thanks, > > > > >>> > > Madhawa > > > > >>> > > > > > > >>> > > > > > > >>> > > On Tue, Oct 18, 2022 at 6:57 PM larry mccay < > > > [email protected] > > > > > > > > > >>> wrote: > > > > >>> > > > > > > >>> > > > Sorry, I missed commenting on this: > > > > >>> > > > > > > > >>> > > > "There is also no concept as an emeritus PPMC > member > > > at the > > > > ASF." > > > > >>> > > > > > > > >>> > > > I assume that we can remove PPMC members that do > not > > > opt-in > > > > >>> explicitly > > > > >>> > at > > > > >>> > > > this point. > > > > >>> > > > They will have every opportunity to rejoin. > > > > >>> > > > > > > > >>> > > > On Tue, Oct 18, 2022 at 12:48 PM larry mccay < > > > > [email protected]> > > > > >>> > wrote: > > > > >>> > > > > > > > >>> > > > > I will ask in a separate thread, @Justin Mclean < > > > > >>> [email protected] > > > > >>> > > > > > > >>> > > - > > > > >>> > > > > thanks. > > > > >>> > > > > Adding JB adds another company and we are > certainly > > > open to > > > > >>> anyone > > > > >>> > else > > > > >>> > > > > that would like to join as a mentor. > > > > >>> > > > > At the end of the day, the mentors are for > > > instilling the > > > > Apache > > > > >>> Way > > > > >>> > > > > knowledge and steering toward graduation. > > > > >>> > > > > I feel that this diversity, while nice to have, > is > > > less > > > > important > > > > >>> > than > > > > >>> > > > > that of the PPMC and committers for the long term > > > health > > > > of the > > > > >>> > > > community. > > > > >>> > > > > > > > > >>> > > > > We need to push this podling to graduation as > > > quickly as > > > > possible > > > > >>> > since > > > > >>> > > > it > > > > >>> > > > > is rather mature and needs to get to the next > > level. > > > > >>> > > > > > > > > >>> > > > > Again, any potential Mentors that would like to > > join > > > are > > > > more > > > > >>> than > > > > >>> > > > welcome. > > > > >>> > > > > > > > > >>> > > > > On Tue, Oct 18, 2022 at 12:38 PM Justin Mclean < > > > > >>> > > [email protected] > > > > >>> > > > > > > > > >>> > > > > wrote: > > > > >>> > > > > > > > > >>> > > > >> Hi, > > > > >>> > > > >> > > > > >>> > > > >> I’m sorry, but Imran Rashid can’t be a mentor > for > > > the > > > > project as > > > > >>> > they > > > > >>> > > > are > > > > >>> > > > >> not an IPMC member. Currently, both Sunil and > > Larry > > > (as > > > > they > > > > >>> are ASF > > > > >>> > > > >> members) need to ask to join the IPMC and NOTICE > > > sent to > > > > the ASF > > > > >>> > > board. > > > > >>> > > > I > > > > >>> > > > >> would also prefer that mentors come from > different > > > > companies. > > > > >>> > > > >> > > > > >>> > > > >> There is also no concept as an emeritus PPMC > > member > > > at > > > > the ASF. > > > > >>> > > > >> > > > > >>> > > > >> Kind Regards, > > > > >>> > > > >> Justin > > > > >>> > > > >> > > > > >>> > > > >> > > > > >>> > > > >> > > > > >>> > > > > > > > -------------------------------------------------------------------- > > - > > > > >>> > > > >> To unsubscribe, e-mail: > > > > >>> [email protected] > > > > >>> > > > >> For additional commands, e-mail: > > > > >>> [email protected] > > > > >>> > > > >> > > > > >>> > > > >> > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > >>> > > > > >> > > > > > > > > > > > > > > -------------------------------------------------------------------- > > - > > > > To unsubscribe, e-mail: [email protected] > > > > For additional commands, e-mail: > [email protected] > > > > > > > > > > > > >
