Re: [Discuss] ​​CEP-35: Add PIP support for CQLSH

[Patrick McFadin]

Dinesh raises some good points.

If we do adopt this, there will be non-zero overhead of the release
process. This is fine but we need volunteers to run this process. My
understanding is that they need to be ideally PMC or at least Committers
on the project to go through all the steps to successfully release a new
artifact for our users.

Which was addressed in the proposed changes part of the CEP:

- A document detailing procedures for releasing to PyPI.org. This document
should include details on:

   1. How release to PyPI can be integrated into the build process. Can
   this be done with automation?
   2. How will credentials, permissions and ownership of packages on PyPI
   be managed?

-
My first thought was automation and integration into the build release.

Can you briefly outline the steps that need to be followed for a PyPI
release, Brad?

Patrick


On Wed, Aug 9, 2023 at 2:54 PM Abe Ratnofsky  wrote:

> I think it would be good for the project to have an official PyPI
> distribution, and the signal from users (40K downloads per month) is a
> clear indication that this is useful. Timely releases would help us get
> future improvements to cqlsh out faster, and moving this to an official
> distribution would protect users against any changes in this volunteer
> effort in case something happens in the future.
>
> +1 (nb)
>
> --
> Abe
>
> On Aug 9, 2023, at 1:33 PM, Brad  wrote:
>
> HI Dinesh,
>
> You are correct that the scope of this CEP is practical, narrow and
> limited to having an official distribution of CQLSH on the official Python
> package repository. Cassandra end-users, who use the CQLSH command line,
> would benefit in several direct ways:
>
>- A timely distribution of new CQLSH versions on the official Python
>package repository aligned with Apache Cassandra releases
>- A trusted distribution overseen by Apache Cassandra instead of third
>party maintainers. Today, there is only trust-based faith that the PyPI
>distribution of CQLSH matches the Apache Open Source one.
>- A lightweight distribution of CQLSH clocking in at 110KB vs
>downloading a 50MB tarball.
>
> Perhaps those are modest goals, but I would suggest they are big wins for
> the Cassandra user community. If you haven't tried it yet, please run '*pip
> install cqlsh*' on your desktop and see how nicely it works. Indeed, the
> return-on-investment of effort here should be really high, as the work is
> mostly already done, it's just run from a private repo at
> https://github.com/jeffwidman/cqlsh and has been maintained continually
> since 2013.
>
> Other initiatives such as subdividing the project(s) or re-writing the
> REPL in another language would be out-of-scope. It would be entirely
> appropriate to have a separate discussion on those two topics, if you wish
> to start that discussion.
>
> The process and degree of overhead required to publish to PyPI will
> require some discovery and discussion. Ideally, it would be possible to
> automate it. That is definitely a topic we need further input from the
> engineers involved in the build-release process.
>
> A pre-CEP discussion of this proposal was started by Jeff on the mailing
> list back in early July, see
> https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d.
>
> Regards,
>
> Brad
>
> On Wed, Aug 9, 2023 at 3:31 PM Dinesh Joshi  wrote:
>
>> Brad,
>>
>> Thanks for starting this discussion. My understanding is that we're
>> simply adding pip support for cqlsh and Apache Cassandra project will
>> officially publish a cqlsh pip package. This is a good goal but other
>> than having an official pip package, what is it that we're gaining?
>> Please don't interpret this as push back on your proposal but I am
>> unclear on what we're trying to solve by making this official
>> distribution. There are several distribution channels and it is
>> untenable to officially support all of them.
>>
>> If we do adopt this, there will be non-zero overhead of the release
>> process. This is fine but we need volunteers to run this process. My
>> understanding is that they need to be ideally PMC or at least Committers
>> on the project to go through all the steps to successfully release a new
>> artifact for our users.
>>
>> I would have liked this CEP to go a bit further than just packaging
>> cqlsh in pip. IMHO we should have cqlsh as a separate sub-project. It
>> doesn't need to live in the cassandra repo. Extracting cqlsh into it's
>> separate repo would allow us to truly decouple cqlsh from the server.
>> This is already true for the most part as we rely on the Python driver
>> which is compatible with several cassandra releases. As it stands today
>> it is not possible for us to update cqlsh without making a Cassandra
>> release.
>>
>> If you truly want to go a bit further, we should consider rewriting
>> cqlsh in Java so we can easily share code from the server. We can then
>> potentially use Java Native Image[1] to produce a truly platform
>> inde

Re: [Discuss] ​​CEP-35: Add PIP support for CQLSH

[Brad]

Distribution is pretty straightforward, the process I follow after 'build'
is complete:

## Release Cassandra CQLSH

You must have the maintainer privilege of `https://pypi.org/project/cqlsh/`


1. verify package locally with pip


pip install -e .



2. Test upload on testpypi



twine upload --repository testpypi dist/*


pip install --index-url https://test.pypi.org/simple/ cqlsh



3. prod upload

twine upload --repository pypi dist/*

Regards,


Brad

On Thu, Aug 10, 2023 at 3:27 PM Patrick McFadin  wrote:

> Dinesh raises some good points.
>
> If we do adopt this, there will be non-zero overhead of the release
> process. This is fine but we need volunteers to run this process. My
> understanding is that they need to be ideally PMC or at least Committers
> on the project to go through all the steps to successfully release a new
> artifact for our users.
>
> Which was addressed in the proposed changes part of the CEP:
>
>
>- A document detailing procedures for releasing to PyPI.org. This
>document should include details on:
>
>
>1. How release to PyPI can be integrated into the build process. Can
>this be done with automation?
>2. How will credentials, permissions and ownership of packages on PyPI
>be managed?
>
>
>-
>
> My first thought was automation and integration into the build release.
>
> Can you briefly outline the steps that need to be followed for a PyPI
> release, Brad?
>
> Patrick
>
>
> On Wed, Aug 9, 2023 at 2:54 PM Abe Ratnofsky  wrote:
>
>> I think it would be good for the project to have an official PyPI
>> distribution, and the signal from users (40K downloads per month) is a
>> clear indication that this is useful. Timely releases would help us get
>> future improvements to cqlsh out faster, and moving this to an official
>> distribution would protect users against any changes in this volunteer
>> effort in case something happens in the future.
>>
>> +1 (nb)
>>
>> --
>> Abe
>>
>> On Aug 9, 2023, at 1:33 PM, Brad  wrote:
>>
>> HI Dinesh,
>>
>> You are correct that the scope of this CEP is practical, narrow and
>> limited to having an official distribution of CQLSH on the official Python
>> package repository. Cassandra end-users, who use the CQLSH command line,
>> would benefit in several direct ways:
>>
>>- A timely distribution of new CQLSH versions on the official Python
>>package repository aligned with Apache Cassandra releases
>>- A trusted distribution overseen by Apache Cassandra instead of
>>third party maintainers. Today, there is only trust-based faith that the
>>PyPI distribution of CQLSH matches the Apache Open Source one.
>>- A lightweight distribution of CQLSH clocking in at 110KB vs
>>downloading a 50MB tarball.
>>
>> Perhaps those are modest goals, but I would suggest they are big wins for
>> the Cassandra user community. If you haven't tried it yet, please run '*pip
>> install cqlsh*' on your desktop and see how nicely it works. Indeed, the
>> return-on-investment of effort here should be really high, as the work is
>> mostly already done, it's just run from a private repo at
>> https://github.com/jeffwidman/cqlsh and has been maintained continually
>> since 2013.
>>
>> Other initiatives such as subdividing the project(s) or re-writing the
>> REPL in another language would be out-of-scope. It would be entirely
>> appropriate to have a separate discussion on those two topics, if you wish
>> to start that discussion.
>>
>> The process and degree of overhead required to publish to PyPI will
>> require some discovery and discussion. Ideally, it would be possible to
>> automate it. That is definitely a topic we need further input from the
>> engineers involved in the build-release process.
>>
>> A pre-CEP discussion of this proposal was started by Jeff on the mailing
>> list back in early July, see
>> https://lists.apache.org/thread/sy3p2b2tncg1bk6x3r0r60y10dm6l18d.
>>
>> Regards,
>>
>> Brad
>>
>> On Wed, Aug 9, 2023 at 3:31 PM Dinesh Joshi  wrote:
>>
>>> Brad,
>>>
>>> Thanks for starting this discussion. My understanding is that we're
>>> simply adding pip support for cqlsh and Apache Cassandra project will
>>> officially publish a cqlsh pip package. This is a good goal but other
>>> than having an official pip package, what is it that we're gaining?
>>> Please don't interpret this as push back on your proposal but I am
>>> unclear on what we're trying to solve by making this official
>>> distribution. There are several distribution channels and it is
>>> untenable to officially support all of them.
>>>
>>> If we do adopt this, there will be non-zero overhead of the release
>>> process. This is fine but we need volunteers to run this process. My
>>> understanding is that they need to be ideally PMC or at least Committers
>>> on the project to go through all the steps to successfully release a new
>>> artifact for our users.
>>>
>>> I would have liked this CEP to go a bit further than just packaging
>>> cqlsh in pip. 

Re: [DISCUSS] Creating a 5.0 landing page

[Hugh Lashbrooke]

The page listing the new 5.0 features is great! The page would list and
link to the upcoming features, as well as have info about upcoming events
where people can learn more about the release (e.g. Contributor Meetings &
Town Halls) and include any existing videos that show more
information. I'll create a Jira ticket and welcome contributions.

In addition to what I first suggested, it would also include a summarised
set of highlights from the release to make it more immediately helpful for
readers before they go deeper.

I'd also love to look into localising it for any available locales, so yes
- that's on the radar here once the page is built.

On Fri, Aug 4, 2023 at 4:05 PM guo Maxwell  wrote:

> Can we have some pages for 5.0 in Chinese？If we can，I can do some help to
> doing some translation ！
> Ekaterina Dimitrova 于2023年8月4日 周五上午10:55写道：
>
>> I honestly didn’t realize events page was added already. Thank you, Josh!
>>
>> So yes, in that case I guess that is what I am asking - shall we add
>> there town halle, etc?
>>
>> On Thu, 3 Aug 2023 at 22:37, Josh McKenzie  wrote:
>>
>>> We actually already have an events page:
>>> https://cassandra.apache.org/_/events.html; not sure if you were saying
>>> we should add one Ekaterina or saying we should add this content there. +1
>>> to the content there and having a landing page that points there +
>>> integrating meetups, town halls, etc.
>>>
>>> Community -> Events on the menu up top in case someone missed it.
>>>
>>> On Thu, Aug 3, 2023, at 4:21 PM, Ekaterina Dimitrova wrote:
>>>
>>>
>>> Hi Hugh,
>>>
>>> Thank you for reaching out. I think this is a great idea. Also, great
>>> timing, considering the community is discussing a potential 5.0 alpha
>>> release soon.
>>>
>>> It seems to me you actually suggest more than one page?
>>> 1) 5.0 and new features - could this be an update of the What’s new
>>> page? -
>>> https://cassandra.apache.org/doc/trunk/cassandra/new/index.html
>>> Adding also links to some of the talks sounds great to me.
>>> 2) Dedicated events page? We were using the Blogs page before but I
>>> don’t think it is a bad idea to split Blog posts from Events page.
>>>
>>> Thank you
>>> Ekaterina
>>>
>>>
>>> On Wed, 2 Aug 2023 at 21:03, Hugh Lashbrooke  wrote:
>>>
>>> With the upcoming release of Apache Cassandra 5.0, I’d like to create a
>>> landing page for the release and what that could look like.
>>>
>>> The landing page would be intended to educate users about what is coming
>>> up in this important release, highlighting why upgrading will be valuable
>>> to them, as well as guiding them into more community activities, such as
>>> Town Halls, where they can learn more and become further involved.
>>>
>>> The 5.0 landing page could include:
>>>
>>>- An overview of the release with a brief summary of the major
>>>features
>>>- A page for each CEP that is likely to be included–with key
>>>features, implementation information, and other technical details. These
>>>pages can also include recordings of relevant Contributor Meetings. Here
>>>is an example for CEP-28 - Spark Bulk Analytics Library
>>>
>>> 
>>>.
>>>- CTAs to community platforms and activities - Slack, Meetups, Town
>>>Halls, Contributor Meetings, etc.
>>>
>>> Let’s discuss! Does this sound valuable? If so, I will create a Jira
>>> ticket and am happy to get started. What other things do you think should
>>> be included in a page like this?
>>>
>>>
>>> --
> you are the apple of my eye !
>


-- 
Hugh Lashbrooke
Director of Community, Constantia.io
LinkedIn  | Twitter
 | Mastodon