I agree that a CEP is a good idea, I'll discuss with Jeff and hope to draft something.
On Wed, Jul 12, 2023 at 6:13 AM Miklosovic, Stefan < stefan.mikloso...@netapp.com> wrote: > CEP is a great idea. The devil is in details and while this looks cool, it > will definitely not hurt to have the nuances ironed out. > > ________________________________________ > From: Patrick McFadin <pmcfa...@gmail.com> > Sent: Tuesday, July 11, 2023 2:24 > To: dev@cassandra.apache.org; German Eichberger > Subject: Re: CASSANDRA-18654 - start publishing CQLSH to PyPI as part of > the release process > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is > safe. > > > > I would say it helps a lot of people. 45k downloads in just last month: > https://pypistats.org/packages/cqlsh<https://pypistats.org/packages/cqlsh> > > I feel like a CEP would be in order, along the lines of CEP-8: > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation > < > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation > > > > Unless anyone objects, I can help you get the CEP together and we can get > a vote, then a JIRA in place for any changes in trunk. > > Patrick > > On Mon, Jul 10, 2023 at 4:58 PM German Eichberger via dev < > dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>> wrote: > Same - really appreciate those efforts and also welcome the upstreaming > and release automation... > > German > ________________________________ > From: Jeff Widman <j...@jeffwidman.com<mailto:j...@jeffwidman.com>> > Sent: Sunday, July 9, 2023 1:44 PM > To: Max C. <mc_cassand...@core43.com<mailto:mc_cassand...@core43.com>> > Cc: dev@cassandra.apache.org<mailto:dev@cassandra.apache.org> < > dev@cassandra.apache.org<mailto:dev@cassandra.apache.org>>; Brad > Schoening <bscho...@gmail.com<mailto:bscho...@gmail.com>> > Subject: [EXTERNAL] Re: CASSANDRA-18654 - start publishing CQLSH to PyPI > as part of the release process > > You don't often get email from j...@jeffwidman.com<mailto: > j...@jeffwidman.com>. Learn why this is important< > https://aka.ms/LearnAboutSenderIdentification> > Thanks Max, always encouraging to hear that the time I spend on open > source is helping others. > > Your use case is very similar to what drove my original desire to get > involved with the project. Being able to `pip install cqlsh` from a dev > machine was so much lighter weight than the alternatives. > > Anyone else care to weigh in on this? > > What are the next steps to move to a decision? > > Cheers, > Jeff > > On Sat, Jul 8, 2023, 7:23 PM Max C. <mc_cassand...@core43.com<mailto: > mc_cassand...@core43.com>> wrote: > > As a user, I really appreciate your efforts Jeff & Brad. I would *love* > for the C* project to officially support this. > > In our environment we have a lot of client machines that all share common > NFS mounted directories. It's much easier for us to create a Python > virtual environment on a file server with the cqlsh PyPI package installed > than it is to install the Cassandra RPMs on every single machine. Before I > discovered your PyPI package, our developers would need to login to a > Cassandra node in order to run cqlsh. The cqlsh PyPI package, however, is > in our standard "python dev tools" virtual environment -- along with > Ansible, black, isort and various other Python packages; which means it's > accessible to everyone, everywhere. > > I agree that this should not replace packaging cqlsh in the Cassandra RPM, > so much provide an additional option for installing cqlsh without the > baggage of installing the full Cassandra package. > > Thanks again for your work Jeff & Brad. > > - Max > > On 7/6/2023 5:55 PM, Jeff Widman wrote: > Myself and Brad Schoening currently maintain > https://pypi.org/project/cqlsh/<https://pypi.org/project/cqlsh/> which > repackages CQLSH that ships with every Cassandra release. > > This way: > > * anyone who wants a lightweight client to talk to a remote cassandra > can simply `pip install cqlsh` without having to download the full > cassandra source, unzip it, etc. > * it's very easy for folks to use it as scaffolding in their python > scripts/tooling since they can simply include it in the list of their > required dependencies. > > We currently handle the packaging by waiting for a release, then manually > copy/pasting the code out of the cassandra source tree into > https://github.com/jeffwidman/cqlsh<https://github.com/jeffwidman/cqlsh> > which has some additional build/python package configuration files, then > using standard python tooling to publish to PyPI. > > Given that our project is simply a build/packaging project, I wanted to > start a conversation about upstreaming this into core Cassandra. I realize > that Cassandra has no interest in maintaining lots of build targets... but > given that cqlsh is written in Python and publishing to PyPI enables DBA's > to share more complicated tooling built on top of it this seems like a > natural fit for core cassandra rather than a standalone project. > > Goal: > When a Cassandra release happens, the build/release process automatically > publishes cqlsh to https://pypi.org/project/cqlsh/< > https://pypi.org/project/cqlsh/>. > > Non-Goal: This is _not_ about having cassandra itself rely on PyPI. There > was some initial chatter about that in > https://issues.apache.org/jira/browse/CASSANDRA-18654< > https://issues.apache.org/jira/browse/CASSANDRA-18654>, but that adds a > lot of complexity, and I'm honestly not sure it's a great idea. Even if > folks later want to go that route, the first hurdle is publishing to PyPI, > so for now let's keep the scope of the discussion limited to treating PyPI > purely as a release target, and not as an ingredient to a release. > > From an implementation perspective, this should be very straightforward. > We don't have any differences from the CQLSH source that's in cassandra, > instead we point folks to make changes to cqlsh in the Cassandra source. In > fact we've made multiple contributions back to `cqlsh` ourselves and have > drastically cleaned up the code: > https://github.com/search?q=repo%3Aapache%2Fcassandra%20is%3Apr%20author%3Ajeffwidman%20author%3Abschoening&type=pullrequests > < > https://github.com/search?q=repo%3Aapache%2Fcassandra%20is%3Apr%20author%3Ajeffwidman%20author%3Abschoening&type=pullrequests>. > So the only real change is adding the package config files and the build / > release pipeline. > > We realize the Cassandra team isn't python/PyPI experts, so we'd be more > than happy to help wire this up and maintain it. I am also a maintainer of > kazoo and kafka-python which are both popular python clients for other > distributed databases. So I'm very familiar with open source, python, and > distributed databases. > > My one hesitation around this discussion is that I'm a little concerned > that we might lose the nimbleness we've currently got from having a > separate project. Ie, if something is screwed up on PyPI / the build > process, we can quickly get it fixed and get a new release out so that > users aren't blocked. Would it be possible as part of this process to > continue that myself/Brad had commit rights to the build process for PyPI? > To be clear, I'm not asking for commit rights to the Java code or anything > outside of Python, I just want to be sure that if we go to the trouble of > working with you to upstream this that there's a commitment from Cassandra > to keeping this build working, or to letting us be able to fix the build. > Otherwise there's no point in upstreaming it only for it to go unmaintained > leaving us looking on helplessly from the sidelines. I'm very flexible here > on the solution. > > Thoughts? > > -- > > Jeff Widman > jeffwidman.com<http://www.jeffwidman.com/> | 740-WIDMAN-J (943-6265) > <>< >
