Dinesh raises some good points. If we do adopt this, there will be non-zero overhead of the release process. This is fine but we need volunteers to run this process. My understanding is that they need to be ideally PMC or at least Committers on the project to go through all the steps to successfully release a new artifact for our users.
Which was addressed in the proposed changes part of the CEP: - A document detailing procedures for releasing to PyPI.org. This document should include details on: 1. How release to PyPI can be integrated into the build process. Can this be done with automation? 2. How will credentials, permissions and ownership of packages on PyPI be managed? - My first thought was automation and integration into the build release. Can you briefly outline the steps that need to be followed for a PyPI release, Brad? Patrick On Wed, Aug 9, 2023 at 2:54 PM Abe Ratnofsky <a...@aber.io> wrote: > I think it would be good for the project to have an official PyPI > distribution, and the signal from users (40K downloads per month) is a > clear indication that this is useful. Timely releases would help us get > future improvements to cqlsh out faster, and moving this to an official > distribution would protect users against any changes in this volunteer > effort in case something happens in the future. > > +1 (nb) > > -- > Abe > > On Aug 9, 2023, at 1:33 PM, Brad <bscho...@gmail.com> wrote: > > HI Dinesh, > > You are correct that the scope of this CEP is practical, narrow and > limited to having an official distribution of CQLSH on the official Python > package repository. Cassandra end-users, who use the CQLSH command line, > would benefit in several direct ways: > > - A timely distribution of new CQLSH versions on the official Python > package repository aligned with Apache Cassandra releases > - A trusted distribution overseen by Apache Cassandra instead of third > party maintainers. Today, there is only trust-based faith that the PyPI > distribution of CQLSH matches the Apache Open Source one. > - A lightweight distribution of CQLSH clocking in at 110KB vs > downloading a 50MB tarball. > > Perhaps those are modest goals, but I would suggest they are big wins for > the Cassandra user community. If you haven't tried it yet, please run '*pip > install cqlsh*' on your desktop and see how nicely it works. Indeed, the > return-on-investment of effort here should be really high, as the work is > mostly already done, it's just run from a private repo at > https://github.com/jeffwidman/cqlsh and has been maintained continually > since 2013. > > Other initiatives such as subdividing the project(s) or re-writing the > REPL in another language would be out-of-scope. It would be entirely > appropriate to have a separate discussion on those two topics, if you wish > to start that discussion. > > The process and degree of overhead required to publish to PyPI will > require some discovery and discussion. Ideally, it would be possible to > automate it. That is definitely a topic we need further input from the > engineers involved in the build-release process. > > A pre-CEP discussion of this proposal was started by Jeff on the mailing > list back in early July, see > https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d. > > Regards, > > Brad > > On Wed, Aug 9, 2023 at 3:31 PM Dinesh Joshi <djo...@apache.org> wrote: > >> Brad, >> >> Thanks for starting this discussion. My understanding is that we're >> simply adding pip support for cqlsh and Apache Cassandra project will >> officially publish a cqlsh pip package. This is a good goal but other >> than having an official pip package, what is it that we're gaining? >> Please don't interpret this as push back on your proposal but I am >> unclear on what we're trying to solve by making this official >> distribution. There are several distribution channels and it is >> untenable to officially support all of them. >> >> If we do adopt this, there will be non-zero overhead of the release >> process. This is fine but we need volunteers to run this process. My >> understanding is that they need to be ideally PMC or at least Committers >> on the project to go through all the steps to successfully release a new >> artifact for our users. >> >> I would have liked this CEP to go a bit further than just packaging >> cqlsh in pip. IMHO we should have cqlsh as a separate sub-project. It >> doesn't need to live in the cassandra repo. Extracting cqlsh into it's >> separate repo would allow us to truly decouple cqlsh from the server. >> This is already true for the most part as we rely on the Python driver >> which is compatible with several cassandra releases. As it stands today >> it is not possible for us to update cqlsh without making a Cassandra >> release. >> >> If you truly want to go a bit further, we should consider rewriting >> cqlsh in Java so we can easily share code from the server. We can then >> potentially use Java Native Image[1] to produce a truly platform >> independent binary like golang. Python has its strengths but it does get >> hairy as it expects certain runtime components on the target. Java With >> Native Image we make things very simple from a user's perspective very >> similar to how golang produces statically linked binaries. This might be >> a very far out thought but it is worth exploring. I believe GraalVM's >> license might allow us to produce binaries that we can incorporate in >> our release but IANAL so maybe we can ask ASF legal on their opinion. >> >> Giving cqlsh it's own identity as a sub-project might help us build a >> roadmap and evolve it along these lines. >> >> I would like other folks to chime in with their opinions. >> >> Dinesh >> >> On 8/9/23 09:18, Brad wrote: >> > >> > As per the CEP process guidelines, I'm starting a formal DISCUSS thread >> > to resume the conversation started here[1]. >> > >> > The developers who maintain the Python CQLSH client on the official >> > Python PYPI repository would like to integrate and donate their open >> > source work to the Apache Cassandra project so it can be more tightly >> > and seamlessly integrated. >> > >> > The Apache Cassandra project pre-dates the adoption in Python 3.4 of >> > PyPI as the default package manager. As a result, an unofficial >> > distribution has been provided by a group of developers who have >> > maintained the repository there since October 2013. >> > >> > The installable version of CQLSH on PyPI.org allows end users to install >> > a cqlsh client with PIP - no tarball or path setup required. I.e., >> > >> > $ pip install cqlsh >> > >> > This popular package has 50K downloads per month and is today maintained >> > by Jeff Wideman and Brad Schoening. The PYPI package is updated upon >> > every major release by simply repackaging the CQLSH that ships with >> > every Cassandra release. >> > >> > CQLSH PyPI Repository: https://pypi.org/project/cqlsh/ >> > <https://pypi.org/project/cqlsh/> >> > >> > >> > This CEP Proposal suggests incorporating PYPI as a regular part of the >> > Cassandra release process and making the CQLSH project on PYPI an >> > official distribution point. >> > >> > The full CEP can be reviewed at: >> > >> > Wiki: CEP-35: Add PIP support for CQLSH >> > < >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425995 >> >. >> > >> > Jira: CASSANDRA-18654 >> > <https://issues.apache.org/jira/browse/CASSANDRA-18654> >> > >> > >> > But in brief, the proposal will: >> > >> > * Add PyPI.org as an official distribution point for CQLSH >> > * Allow end users to install CQLSH with simply 'pip install cqlsh' on >> > MacOS, Windows and Linux platforms. >> > * Donate the modest amount of existing configuration files by the >> > authors to Apache Cassandra >> > * This only involves the Python CQLSH client, no changes to >> > distribution of Java server side code and tools are involved. >> > >> > We welcome further discussion and suggestions regarding this proposal on >> > the mailing list here. >> > >> > Regards, >> > >> > Jeff Widman & >> > Brad Schoening >> > >> > [1] https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d >> > < >> https://lists.apache.org/thread.html/ra7caa1dd42ccaa04bcabfbc33233995c125c655f9a3cdb2c7bd8e9f7%40%3Cdev.cassandra.apache.org%3E >> > >> >> >
