Re: Status of CEP-1

[Francisco Guerrero]

Hi folks,

Thanks for all the input. I'm trying to gather all the comments, and from what I
can gather it seems that most of the opinions are converging towards scoping
a Sidecar release. These are the items that were called out that we will need
for a release:
- Documentation
- Authorization / Authentication
- vnode support

There are some smaller bug fixes that need to be included that we can label
as part of the 1.0 release.[1][2][3]

If there are any other tasks we need to completed, I encourage the community
to create JIRAs that can be in the release milestone for the Sidecar.

I agree with Stefan that OpenAPI is desirable. OpenAPI is something I've been
looking into as well. I'm also glad to see Abhijeet can help with the
documentation effort, I can also help on that front.

Hopefully, I've captured your comments as truly as possible.

Thanks again for all the feedback.

Best,
- Francisco

[1] https://issues.apache.org/jira/browse/CASSANDRASC-120
[2] https://issues.apache.org/jira/browse/CASSANDRASC-121
[3] https://issues.apache.org/jira/browse/CASSANDRASC-122

On 2024/10/02 15:29:20 Jon Haddad wrote:
> When I developed some of the original sidecar code, it was based on REST
> Easy, which would have allowed us to generate the spec automatically.  I
> did this in a similar project.
> 
> That was removed here:
> https://issues.apache.org/jira/browse/CASSANDRASC-57
> 
> But unfortunately it looks like you can't generate the spec from the code.
> 
> Disappointing, that functionality was really useful.  I wish someone had
> asked me before gutting it.
> 
> Jon
> 
> 
> 
> On Wed, Oct 2, 2024 at 11:16 AM Štefan Miklošovič 
> wrote:
> 
> > Something like this:
> >
> > https://instaclustr.github.io/instaclustr-icarus-go-client/
> >
> > On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič 
> > wrote:
> >
> >> While documenting endpoints please use something like OpenAPI
> >> specification. The sidecar should expose this document itself so when I go
> >> to this and that URL, I see all endpoints, I put the payloads / parameters
> >> for them and I can just directly call that, no messing with curl / wget or
> >> programmatically or whatever like that. The barrier to exercise the basic
> >> functionality has to virtually not be there.
> >>
> >> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey 
> >> wrote:
> >>
> >>> Hi folks,
> >>>
> >>> I have been using Sidecar recently and have found some of its
> >>> functionalities to be quite useful. Hari and I are also working on CEP-40
> >>> which aims to introduce live migration features in Sidecar in the
> >>> near future.
> >>>
> >>> However, as others have mentioned, I agree that it currently lacks
> >>> proper documentation.
> >>>
> >>> Since this is an official Apache project, I believe that creating
> >>> comprehensive documentation would be beneficial. This documentation should
> >>> include an overview, architecture, a list and description of various
> >>> endpoints, and some examples or tutorials on how to use Sidecar's 
> >>> features.
> >>>
> >>> This documentation would help people get started with Sidecar and lower
> >>> the entry barrier for many. We can update the documentation incrementally
> >>> as needed, along with future enhancements and new features. However,
> >>> creating some form of formal documentation would be very helpful.
> >>>
> >>> To this end I'm willing and highly interested in writing some form of
> >>> formal documentation for the Sidecar project. Please let me know your
> >>> thoughts/opinions on this proposal.
> >>>
> >>>
> >>>
> >>> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič 
> >>> wrote:
> >>>
>  Totally agree with Jon here basically on all fronts. Apache Cassandra
>  Sidecar was always a hard nut to crack for me, that is probably why I 
>  have
>  not been involved with that a lot even that is a great tool to have and 
>  be
>  invested in as I was writing my own sidecar and I found a lot of
>  similarities and problems Apache's sidecar tries to fix. There was some
>  invisible barrier I have never managed to jump over. I was looking around
>  and I am very sorry if I just have not found it yet but there is not a 
>  list
>  of endpoints a sidecar has, is there? In readme and dev docs there is 
>  just
>  nothing. Taking it at a face value I just don't know what Sidecar is
>  capable of and how to use it. I see in the commit history there is a 
>  bunch
>  of commits mentioning S3 but it is a total blackbox for me as a potential
>  user.
> 
>  On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad 
>  wrote:
> 
> > I don't think we should release sidecar 1.0 without any docs.
> >
> > I took a look through the closed JIRAs to see what's there.  Here's
> > what I found, please correct me if there's more:
> >
> > - Lots of stuff related to analytics.
> >
> > I would be pretty excited for this, but the analytics library only
> > wo

Re: [DISCUSS] Modeling JIRA fix version for subprojects

[Yifan Cai]

+1 on all the points raised by Mick. Please let me know if there is
anything I can help with.

- Yifan

On Wed, Oct 2, 2024 at 8:13 AM Josh McKenzie  wrote:

> - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as
> in any subproject channels
> - some cadence of dev@ ML updates, e.g. on activities, or dependency
> changes, etc
> - regular releases
>
> Agree on all 3 points. Also - I've *definitely* fallen off on the project
> updates for mainline; I'll pick that back up after ApacheCon.
>
>
> On Wed, Oct 2, 2024, at 1:57 AM, Mick Semb Wever wrote:
>
> To play devil's advocate here, it's important that the subprojects don't
> lose visibility and silo from the rest of the project.
>
> There are different ways to solve this, and lumping everything into one
> jira project is a messy and poor way of doing it.  But as the sidecar has
> shown us, subproject activity should somehow be made noisy to us.  We need
> sorts of common spaces in the project.
>
> If we go the separate jira project route, then some suggestions to help
> with this are:
> - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as
> in any subproject channels
> - some cadence of dev@ ML updates, e.g. on activities, or dependency
> changes, etc
> - regular releases
>
>
> On Tue, 9 Apr 2024 at 04:11, Dinesh Joshi  wrote:
>
> hi folks - sorry to have dropped the ball on responding to this thread.
>
> My 2 cents are as follows -
>
> 1. Having a separate JIRA project for each sub-project will add management
> overhead. This option, however, allows us to model unique workflows for the
> sub-project.
>
> 2. Managing the sub-project as part of the Cassandra JIRA project would
> imply less management overhead but the sub-project would need to conform to
> the same workflows.
>
> I would pick option 1 unless there is a strong reason and desire to manage
> a separate Jira project. We can always split out the Java Driver project if
> things don't work out. OTOH merging a Jira project is harder.
>
> Thanks,
>
> Dinesh
>
> On Thu, Apr 4, 2024 at 12:45 PM Abe Ratnofsky  wrote:
>
> CEP-8 proposes using separate Jira projects per Cassandra sub-project:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation
>
> > We suggest distinct Jira projects, one per driver, all to be created.
>
> I don't see any discussion changing that from the [DISCUSS] or vote
> threads:
> https://lists.apache.org/thread/01pljcncyjyo467l5orh8nf9okrh7oxm
> https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
> https://lists.apache.org/thread/crolkrhd4y6tt3k4hsy204xomshlcp4p
>
> But looks like upon acceptance that was changed:
> https://lists.apache.org/thread/dhov01s8dvvh3882oxhkmmfv4tqdd68o
>
> > New issues will be tracked under the CASSANDRA project on Apache’s JIRA <
> https://issues.apache.org/jira/projects/CASSANDRA> under the component
> ‘Client/java-driver’.
>
> I'm in favor of using the same Jira as Cassandra proper. Committership is
> project-wide, so having a standardized process (same ticket flow, review
> rules, labels, etc. is beneficial). But multiple votes happened based on
> the content of the CEP, so we should stick to what was voted on and move to
> a separate Jira.
>
> --
> Abe
>
>
>

Re: Status of CEP-1

[Francisco Guerrero]

> By vnode support, do you mean the analytics library?  Or do other features
> in sidecar not work with vnodes?

Mostly Analytics, which should not be a blocker for Sidecar. However, I do
feel there should be more testing around vnodes in Sidecar.

> If we're talking about analytics, that gets a little complicated.  Are we
> also then talking about 1.0'ing analytics?  If so, I think we need support
> for 5.0 and BTI there.

We need to release Analytics at some point as well, we should have a similar
discuss thread regarding Analytics. We definitely need to support 5.0 for the
Analytics release, but that's orthogonal to Sidecar.

> Increasing the size of the ecosystem is challenging...

Yeah, definitely. I agree that we need to support the latest released version
of Cassandra in ecosystem projects. However, without official releases
there won't be adoption and without adoption there won't be feedback from
the community on what features / improvements are needed.

On 2024/10/02 18:05:42 Jon Haddad wrote:
> By vnode support, do you mean the analytics library?  Or do other features
> in sidecar not work with vnodes?
> 
> If we're talking about analytics, that gets a little complicated.  Are we
> also then talking about 1.0'ing analytics?  If so, I think we need support
> for 5.0 and BTI there.
> 
> In my opinion, if something core to the project has official releases, such
> as drivers or operational tooling that people depend on, it should also
> support the latest version of Cassandra, on release.  It doesn't look good
> if we release C* without usable drivers or tooling to operate it.  It is
> massively deflating to announce we just released 5.0 (exciting!) but you
> can't use it for 6 months because you're using analytics lib and the people
> that contribute to it has better things to do with their time.  I think
> part of the obligation of maintaining these projects needs to be keeping up
> with latest releases.  If that can't be done, we should continue to treat
> it as a use-as-your-own-risk thing without official releases.
> 
> Increasing the size of the ecosystem is challenging...
> 
> Jon
> 
> 
> 
> 
> On Wed, Oct 2, 2024 at 1:21 PM Francisco Guerrero 
> wrote:
> 
> > Hi folks,
> >
> > Thanks for all the input. I'm trying to gather all the comments, and from
> > what I
> > can gather it seems that most of the opinions are converging towards
> > scoping
> > a Sidecar release. These are the items that were called out that we will
> > need
> > for a release:
> > - Documentation
> > - Authorization / Authentication
> > - vnode support
> >
> > There are some smaller bug fixes that need to be included that we can label
> > as part of the 1.0 release.[1][2][3]
> >
> > If there are any other tasks we need to completed, I encourage the
> > community
> > to create JIRAs that can be in the release milestone for the Sidecar.
> >
> > I agree with Stefan that OpenAPI is desirable. OpenAPI is something I've
> > been
> > looking into as well. I'm also glad to see Abhijeet can help with the
> > documentation effort, I can also help on that front.
> >
> > Hopefully, I've captured your comments as truly as possible.
> >
> > Thanks again for all the feedback.
> >
> > Best,
> > - Francisco
> >
> > [1] https://issues.apache.org/jira/browse/CASSANDRASC-120
> > [2] https://issues.apache.org/jira/browse/CASSANDRASC-121
> > [3] https://issues.apache.org/jira/browse/CASSANDRASC-122
> >
> > On 2024/10/02 15:29:20 Jon Haddad wrote:
> > > When I developed some of the original sidecar code, it was based on REST
> > > Easy, which would have allowed us to generate the spec automatically.  I
> > > did this in a similar project.
> > >
> > > That was removed here:
> > > https://issues.apache.org/jira/browse/CASSANDRASC-57
> > >
> > > But unfortunately it looks like you can't generate the spec from the
> > code.
> > >
> > > Disappointing, that functionality was really useful.  I wish someone had
> > > asked me before gutting it.
> > >
> > > Jon
> > >
> > >
> > >
> > > On Wed, Oct 2, 2024 at 11:16 AM Štefan Miklošovič <
> > smikloso...@apache.org>
> > > wrote:
> > >
> > > > Something like this:
> > > >
> > > > https://instaclustr.github.io/instaclustr-icarus-go-client/
> > > >
> > > > On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič <
> > smikloso...@apache.org>
> > > > wrote:
> > > >
> > > >> While documenting endpoints please use something like OpenAPI
> > > >> specification. The sidecar should expose this document itself so when
> > I go
> > > >> to this and that URL, I see all endpoints, I put the payloads /
> > parameters
> > > >> for them and I can just directly call that, no messing with curl /
> > wget or
> > > >> programmatically or whatever like that. The barrier to exercise the
> > basic
> > > >> functionality has to virtually not be there.
> > > >>
> > > >> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey <
> > dubey.abhijee...@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Hi folks,
> > > >>>
> > > >>> I have been using Sidecar 

Re: [DISCUSS] Modeling JIRA fix version for subprojects

[Brandon Williams]

I think we just need to ask infra to create the jira instances, but I
guess we need to have some kind of consistent naming scheme to help
identify them?

Kind Regards,
Brandon

On Wed, Oct 2, 2024 at 1:02 PM Francisco Guerrero  wrote:
>
> +1 too on the points brought by Mick, we need more visibility into
> subprojects. For starters, we should look into integrating Qbot
> notifications in #cassandra-dev and #cassandra-noise for
> CASSANDRASC tickets. Let me know if I can help with that.
>
> On 2024/10/02 17:39:28 Yifan Cai wrote:
> > +1 on all the points raised by Mick. Please let me know if there is
> > anything I can help with.
> >
> > - Yifan
> >
> > On Wed, Oct 2, 2024 at 8:13 AM Josh McKenzie  wrote:
> >
> > > - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as
> > > in any subproject channels
> > > - some cadence of dev@ ML updates, e.g. on activities, or dependency
> > > changes, etc
> > > - regular releases
> > >
> > > Agree on all 3 points. Also - I've *definitely* fallen off on the project
> > > updates for mainline; I'll pick that back up after ApacheCon.
> > >
> > >
> > > On Wed, Oct 2, 2024, at 1:57 AM, Mick Semb Wever wrote:
> > >
> > > To play devil's advocate here, it's important that the subprojects don't
> > > lose visibility and silo from the rest of the project.
> > >
> > > There are different ways to solve this, and lumping everything into one
> > > jira project is a messy and poor way of doing it.  But as the sidecar has
> > > shown us, subproject activity should somehow be made noisy to us.  We need
> > > sorts of common spaces in the project.
> > >
> > > If we go the separate jira project route, then some suggestions to help
> > > with this are:
> > > - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as
> > > in any subproject channels
> > > - some cadence of dev@ ML updates, e.g. on activities, or dependency
> > > changes, etc
> > > - regular releases
> > >
> > >
> > > On Tue, 9 Apr 2024 at 04:11, Dinesh Joshi  wrote:
> > >
> > > hi folks - sorry to have dropped the ball on responding to this thread.
> > >
> > > My 2 cents are as follows -
> > >
> > > 1. Having a separate JIRA project for each sub-project will add management
> > > overhead. This option, however, allows us to model unique workflows for 
> > > the
> > > sub-project.
> > >
> > > 2. Managing the sub-project as part of the Cassandra JIRA project would
> > > imply less management overhead but the sub-project would need to conform 
> > > to
> > > the same workflows.
> > >
> > > I would pick option 1 unless there is a strong reason and desire to manage
> > > a separate Jira project. We can always split out the Java Driver project 
> > > if
> > > things don't work out. OTOH merging a Jira project is harder.
> > >
> > > Thanks,
> > >
> > > Dinesh
> > >
> > > On Thu, Apr 4, 2024 at 12:45 PM Abe Ratnofsky  wrote:
> > >
> > > CEP-8 proposes using separate Jira projects per Cassandra sub-project:
> > >
> > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation
> > >
> > > > We suggest distinct Jira projects, one per driver, all to be created.
> > >
> > > I don't see any discussion changing that from the [DISCUSS] or vote
> > > threads:
> > > https://lists.apache.org/thread/01pljcncyjyo467l5orh8nf9okrh7oxm
> > > https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
> > > https://lists.apache.org/thread/crolkrhd4y6tt3k4hsy204xomshlcp4p
> > >
> > > But looks like upon acceptance that was changed:
> > > https://lists.apache.org/thread/dhov01s8dvvh3882oxhkmmfv4tqdd68o
> > >
> > > > New issues will be tracked under the CASSANDRA project on Apache’s JIRA 
> > > > <
> > > https://issues.apache.org/jira/projects/CASSANDRA> under the component
> > > ‘Client/java-driver’.
> > >
> > > I'm in favor of using the same Jira as Cassandra proper. Committership is
> > > project-wide, so having a standardized process (same ticket flow, review
> > > rules, labels, etc. is beneficial). But multiple votes happened based on
> > > the content of the CEP, so we should stick to what was voted on and move 
> > > to
> > > a separate Jira.
> > >
> > > --
> > > Abe
> > >
> > >
> > >
> >

Re: Status of CEP-1

[Jon Haddad]

I don't think we should release sidecar 1.0 without any docs.

I took a look through the closed JIRAs to see what's there.  Here's what I
found, please correct me if there's more:

- Lots of stuff related to analytics.

I would be pretty excited for this, but the analytics library only works
with single token clusters.  Most folks don't run Cassandra this way.  I
realize there's some element of everyone needs to scratch their own itch,
but I don't think we can really call this a useful feature if the
overwhelming majority of folks can't use it.  I've worked with a couple
hundred teams over the years and can only think of 1 org outside of Apple
and Netflix that used 1 token, and It was a cluster that predated v-nodes.

The analytics repo says it's compatible with Cassandra 4, but not 5.

- Backup & Restore from S3

Is this compatible with other cloud providers or object stores?  It
specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
correct in reading it supports backing up snapshots, no continuous
backups?  Seems like we should have at least feature parity with Medusa if
we're going to release something here.

All the other closed JIRAs look related to these two items.  So the
question is, are we releasing 1.0 as an limited S3 backup and restore
tool?  One that prevents you from upgrading to Cassandra 5 if you happen to
use single token clusters?

Who is the target audience?

Jon



On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi  wrote:

> Currently the Sidecar has a lot of functionality that is immediately
> usable by the community. Apart from minor fixes, the AuthN/Z story would be
> wrapped up soon. Post this, I would propose moving forward with cutting a
> release with the existing feature set so we can get this in the hands of
> our community.
>
> On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell  wrote:
>
>> Have the same question ： what ‘s the plan ？
>>
>> Jeff Jirsa 于2024年10月2日 周三上午10:43写道：
>>
>>>
>>>
>>> On Oct 1, 2024, at 7:26 PM, Josh McKenzie  wrote:
>>>
>>> However it is used by a number of other features as a dependency such as
>>> analytics, backup/restore, repair, metrics, and CDC
>>>
>>> It seems like a natural pressure relief valve for moving operations out
>>> of a core C* node that are well served out of process.
>>>
>>>
>>> Yea, but the point of the foundation is to RELEASE software for the
>>> public good, and the link asserting consensus was dec2018, so its’ 5.5
>>> years and no releases.
>>>
>>> What’s the plan here?
>>>
>>>
>>>
>>>
>>>

Re: Status of CEP-1

[Jon Haddad]

When I developed some of the original sidecar code, it was based on REST
Easy, which would have allowed us to generate the spec automatically.  I
did this in a similar project.

That was removed here:
https://issues.apache.org/jira/browse/CASSANDRASC-57

But unfortunately it looks like you can't generate the spec from the code.

Disappointing, that functionality was really useful.  I wish someone had
asked me before gutting it.

Jon



On Wed, Oct 2, 2024 at 11:16 AM Štefan Miklošovič 
wrote:

> Something like this:
>
> https://instaclustr.github.io/instaclustr-icarus-go-client/
>
> On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič 
> wrote:
>
>> While documenting endpoints please use something like OpenAPI
>> specification. The sidecar should expose this document itself so when I go
>> to this and that URL, I see all endpoints, I put the payloads / parameters
>> for them and I can just directly call that, no messing with curl / wget or
>> programmatically or whatever like that. The barrier to exercise the basic
>> functionality has to virtually not be there.
>>
>> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey 
>> wrote:
>>
>>> Hi folks,
>>>
>>> I have been using Sidecar recently and have found some of its
>>> functionalities to be quite useful. Hari and I are also working on CEP-40
>>> which aims to introduce live migration features in Sidecar in the
>>> near future.
>>>
>>> However, as others have mentioned, I agree that it currently lacks
>>> proper documentation.
>>>
>>> Since this is an official Apache project, I believe that creating
>>> comprehensive documentation would be beneficial. This documentation should
>>> include an overview, architecture, a list and description of various
>>> endpoints, and some examples or tutorials on how to use Sidecar's features.
>>>
>>> This documentation would help people get started with Sidecar and lower
>>> the entry barrier for many. We can update the documentation incrementally
>>> as needed, along with future enhancements and new features. However,
>>> creating some form of formal documentation would be very helpful.
>>>
>>> To this end I'm willing and highly interested in writing some form of
>>> formal documentation for the Sidecar project. Please let me know your
>>> thoughts/opinions on this proposal.
>>>
>>>
>>>
>>> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič 
>>> wrote:
>>>
 Totally agree with Jon here basically on all fronts. Apache Cassandra
 Sidecar was always a hard nut to crack for me, that is probably why I have
 not been involved with that a lot even that is a great tool to have and be
 invested in as I was writing my own sidecar and I found a lot of
 similarities and problems Apache's sidecar tries to fix. There was some
 invisible barrier I have never managed to jump over. I was looking around
 and I am very sorry if I just have not found it yet but there is not a list
 of endpoints a sidecar has, is there? In readme and dev docs there is just
 nothing. Taking it at a face value I just don't know what Sidecar is
 capable of and how to use it. I see in the commit history there is a bunch
 of commits mentioning S3 but it is a total blackbox for me as a potential
 user.

 On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad 
 wrote:

> I don't think we should release sidecar 1.0 without any docs.
>
> I took a look through the closed JIRAs to see what's there.  Here's
> what I found, please correct me if there's more:
>
> - Lots of stuff related to analytics.
>
> I would be pretty excited for this, but the analytics library only
> works with single token clusters.  Most folks don't run Cassandra this
> way.  I realize there's some element of everyone needs to scratch their 
> own
> itch, but I don't think we can really call this a useful feature if the
> overwhelming majority of folks can't use it.  I've worked with a couple
> hundred teams over the years and can only think of 1 org outside of Apple
> and Netflix that used 1 token, and It was a cluster that predated v-nodes.
>
> The analytics repo says it's compatible with Cassandra 4, but not 5.
>
> - Backup & Restore from S3
>
> Is this compatible with other cloud providers or object stores?  It
> specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
> correct in reading it supports backing up snapshots, no continuous
> backups?  Seems like we should have at least feature parity with Medusa if
> we're going to release something here.
>
> All the other closed JIRAs look related to these two items.  So the
> question is, are we releasing 1.0 as an limited S3 backup and restore
> tool?  One that prevents you from upgrading to Cassandra 5 if you happen 
> to
> use single token clusters?
>
> Who is the target audience?
>
> Jon
>
>
>
> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Jo

Re: [DISCUSS] Modeling JIRA fix version for subprojects

[Francisco Guerrero]

+1 too on the points brought by Mick, we need more visibility into
subprojects. For starters, we should look into integrating Qbot
notifications in #cassandra-dev and #cassandra-noise for
CASSANDRASC tickets. Let me know if I can help with that.

On 2024/10/02 17:39:28 Yifan Cai wrote:
> +1 on all the points raised by Mick. Please let me know if there is
> anything I can help with.
> 
> - Yifan
> 
> On Wed, Oct 2, 2024 at 8:13 AM Josh McKenzie  wrote:
> 
> > - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as
> > in any subproject channels
> > - some cadence of dev@ ML updates, e.g. on activities, or dependency
> > changes, etc
> > - regular releases
> >
> > Agree on all 3 points. Also - I've *definitely* fallen off on the project
> > updates for mainline; I'll pick that back up after ApacheCon.
> >
> >
> > On Wed, Oct 2, 2024, at 1:57 AM, Mick Semb Wever wrote:
> >
> > To play devil's advocate here, it's important that the subprojects don't
> > lose visibility and silo from the rest of the project.
> >
> > There are different ways to solve this, and lumping everything into one
> > jira project is a messy and poor way of doing it.  But as the sidecar has
> > shown us, subproject activity should somehow be made noisy to us.  We need
> > sorts of common spaces in the project.
> >
> > If we go the separate jira project route, then some suggestions to help
> > with this are:
> > - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as
> > in any subproject channels
> > - some cadence of dev@ ML updates, e.g. on activities, or dependency
> > changes, etc
> > - regular releases
> >
> >
> > On Tue, 9 Apr 2024 at 04:11, Dinesh Joshi  wrote:
> >
> > hi folks - sorry to have dropped the ball on responding to this thread.
> >
> > My 2 cents are as follows -
> >
> > 1. Having a separate JIRA project for each sub-project will add management
> > overhead. This option, however, allows us to model unique workflows for the
> > sub-project.
> >
> > 2. Managing the sub-project as part of the Cassandra JIRA project would
> > imply less management overhead but the sub-project would need to conform to
> > the same workflows.
> >
> > I would pick option 1 unless there is a strong reason and desire to manage
> > a separate Jira project. We can always split out the Java Driver project if
> > things don't work out. OTOH merging a Jira project is harder.
> >
> > Thanks,
> >
> > Dinesh
> >
> > On Thu, Apr 4, 2024 at 12:45 PM Abe Ratnofsky  wrote:
> >
> > CEP-8 proposes using separate Jira projects per Cassandra sub-project:
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation
> >
> > > We suggest distinct Jira projects, one per driver, all to be created.
> >
> > I don't see any discussion changing that from the [DISCUSS] or vote
> > threads:
> > https://lists.apache.org/thread/01pljcncyjyo467l5orh8nf9okrh7oxm
> > https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
> > https://lists.apache.org/thread/crolkrhd4y6tt3k4hsy204xomshlcp4p
> >
> > But looks like upon acceptance that was changed:
> > https://lists.apache.org/thread/dhov01s8dvvh3882oxhkmmfv4tqdd68o
> >
> > > New issues will be tracked under the CASSANDRA project on Apache’s JIRA <
> > https://issues.apache.org/jira/projects/CASSANDRA> under the component
> > ‘Client/java-driver’.
> >
> > I'm in favor of using the same Jira as Cassandra proper. Committership is
> > project-wide, so having a standardized process (same ticket flow, review
> > rules, labels, etc. is beneficial). But multiple votes happened based on
> > the content of the CEP, so we should stick to what was voted on and move to
> > a separate Jira.
> >
> > --
> > Abe
> >
> >
> >
> 

Re: Status of CEP-1

[Jon Haddad]

By vnode support, do you mean the analytics library?  Or do other features
in sidecar not work with vnodes?

If we're talking about analytics, that gets a little complicated.  Are we
also then talking about 1.0'ing analytics?  If so, I think we need support
for 5.0 and BTI there.

In my opinion, if something core to the project has official releases, such
as drivers or operational tooling that people depend on, it should also
support the latest version of Cassandra, on release.  It doesn't look good
if we release C* without usable drivers or tooling to operate it.  It is
massively deflating to announce we just released 5.0 (exciting!) but you
can't use it for 6 months because you're using analytics lib and the people
that contribute to it has better things to do with their time.  I think
part of the obligation of maintaining these projects needs to be keeping up
with latest releases.  If that can't be done, we should continue to treat
it as a use-as-your-own-risk thing without official releases.

Increasing the size of the ecosystem is challenging...

Jon




On Wed, Oct 2, 2024 at 1:21 PM Francisco Guerrero 
wrote:

> Hi folks,
>
> Thanks for all the input. I'm trying to gather all the comments, and from
> what I
> can gather it seems that most of the opinions are converging towards
> scoping
> a Sidecar release. These are the items that were called out that we will
> need
> for a release:
> - Documentation
> - Authorization / Authentication
> - vnode support
>
> There are some smaller bug fixes that need to be included that we can label
> as part of the 1.0 release.[1][2][3]
>
> If there are any other tasks we need to completed, I encourage the
> community
> to create JIRAs that can be in the release milestone for the Sidecar.
>
> I agree with Stefan that OpenAPI is desirable. OpenAPI is something I've
> been
> looking into as well. I'm also glad to see Abhijeet can help with the
> documentation effort, I can also help on that front.
>
> Hopefully, I've captured your comments as truly as possible.
>
> Thanks again for all the feedback.
>
> Best,
> - Francisco
>
> [1] https://issues.apache.org/jira/browse/CASSANDRASC-120
> [2] https://issues.apache.org/jira/browse/CASSANDRASC-121
> [3] https://issues.apache.org/jira/browse/CASSANDRASC-122
>
> On 2024/10/02 15:29:20 Jon Haddad wrote:
> > When I developed some of the original sidecar code, it was based on REST
> > Easy, which would have allowed us to generate the spec automatically.  I
> > did this in a similar project.
> >
> > That was removed here:
> > https://issues.apache.org/jira/browse/CASSANDRASC-57
> >
> > But unfortunately it looks like you can't generate the spec from the
> code.
> >
> > Disappointing, that functionality was really useful.  I wish someone had
> > asked me before gutting it.
> >
> > Jon
> >
> >
> >
> > On Wed, Oct 2, 2024 at 11:16 AM Štefan Miklošovič <
> smikloso...@apache.org>
> > wrote:
> >
> > > Something like this:
> > >
> > > https://instaclustr.github.io/instaclustr-icarus-go-client/
> > >
> > > On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič <
> smikloso...@apache.org>
> > > wrote:
> > >
> > >> While documenting endpoints please use something like OpenAPI
> > >> specification. The sidecar should expose this document itself so when
> I go
> > >> to this and that URL, I see all endpoints, I put the payloads /
> parameters
> > >> for them and I can just directly call that, no messing with curl /
> wget or
> > >> programmatically or whatever like that. The barrier to exercise the
> basic
> > >> functionality has to virtually not be there.
> > >>
> > >> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey <
> dubey.abhijee...@gmail.com>
> > >> wrote:
> > >>
> > >>> Hi folks,
> > >>>
> > >>> I have been using Sidecar recently and have found some of its
> > >>> functionalities to be quite useful. Hari and I are also working on
> CEP-40
> > >>> which aims to introduce live migration features in Sidecar in the
> > >>> near future.
> > >>>
> > >>> However, as others have mentioned, I agree that it currently lacks
> > >>> proper documentation.
> > >>>
> > >>> Since this is an official Apache project, I believe that creating
> > >>> comprehensive documentation would be beneficial. This documentation
> should
> > >>> include an overview, architecture, a list and description of various
> > >>> endpoints, and some examples or tutorials on how to use Sidecar's
> features.
> > >>>
> > >>> This documentation would help people get started with Sidecar and
> lower
> > >>> the entry barrier for many. We can update the documentation
> incrementally
> > >>> as needed, along with future enhancements and new features. However,
> > >>> creating some form of formal documentation would be very helpful.
> > >>>
> > >>> To this end I'm willing and highly interested in writing some form of
> > >>> formal documentation for the Sidecar project. Please let me know your
> > >>> thoughts/opinions on this proposal.
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Oct 2, 2024

Re: Status of CEP-1

[Jon Haddad]

> Mostly Analytics, which should not be a blocker for Sidecar.

Yes, agreed.  I'm just trying to understand the context of the vnode
statement and how we're framing 1.0 sidecar.

> We definitely need to support 5.0 for the Analytics release, but that's
orthogonal to Sidecar.

It is, unless we're endorsing the analytics library as a primary reason to
use sidecar.  Then it becomes a dependency people rely on, and I don't want
it blocking people from upgrading.

> Yeah, definitely. I agree that we need to support the latest released
version of Cassandra in ecosystem projects. However, without official
releases there won't be adoption and without adoption there won't be
feedback from the community on what features / improvements are needed.

I don't expect that our first release will be feature complete, but it
should be at least compelling.  I'm still not aware of what functionality
exists that would meet that requirement.

Think about this from the perspective of reading a blog post.  We're
excited to announce sidecar 1.0!  Here's what you can do:

1. Backup / restore?
2. ?
3. ?

All I'm asking for are 3 reasons why people should care.  If one of them is
backups, do we provide more flexible backup targets than S3, and if we
provide incremental / continuous backup options?  Is there a scheduler or
do people need to roll their own?  Is it coordinated, or is the intent that
people handle it on their own?

I work with a lot of end users - this is the type of thing that affects
people and can either help or harm the project image.

Jon


On Wed, Oct 2, 2024 at 2:40 PM Francisco Guerrero 
wrote:

> > By vnode support, do you mean the analytics library?  Or do other
> features
> > in sidecar not work with vnodes?
>
> Mostly Analytics, which should not be a blocker for Sidecar. However, I do
> feel there should be more testing around vnodes in Sidecar.
>
> > If we're talking about analytics, that gets a little complicated.  Are we
> > also then talking about 1.0'ing analytics?  If so, I think we need
> support
> > for 5.0 and BTI there.
>
> We need to release Analytics at some point as well, we should have a
> similar
> discuss thread regarding Analytics. We definitely need to support 5.0 for
> the
> Analytics release, but that's orthogonal to Sidecar.
>
> > Increasing the size of the ecosystem is challenging...
>
> Yeah, definitely. I agree that we need to support the latest released
> version
> of Cassandra in ecosystem projects. However, without official releases
> there won't be adoption and without adoption there won't be feedback from
> the community on what features / improvements are needed.
>
> On 2024/10/02 18:05:42 Jon Haddad wrote:
> > By vnode support, do you mean the analytics library?  Or do other
> features
> > in sidecar not work with vnodes?
> >
> > If we're talking about analytics, that gets a little complicated.  Are we
> > also then talking about 1.0'ing analytics?  If so, I think we need
> support
> > for 5.0 and BTI there.
> >
> > In my opinion, if something core to the project has official releases,
> such
> > as drivers or operational tooling that people depend on, it should also
> > support the latest version of Cassandra, on release.  It doesn't look
> good
> > if we release C* without usable drivers or tooling to operate it.  It is
> > massively deflating to announce we just released 5.0 (exciting!) but you
> > can't use it for 6 months because you're using analytics lib and the
> people
> > that contribute to it has better things to do with their time.  I think
> > part of the obligation of maintaining these projects needs to be keeping
> up
> > with latest releases.  If that can't be done, we should continue to treat
> > it as a use-as-your-own-risk thing without official releases.
> >
> > Increasing the size of the ecosystem is challenging...
> >
> > Jon
> >
> >
> >
> >
> > On Wed, Oct 2, 2024 at 1:21 PM Francisco Guerrero 
> > wrote:
> >
> > > Hi folks,
> > >
> > > Thanks for all the input. I'm trying to gather all the comments, and
> from
> > > what I
> > > can gather it seems that most of the opinions are converging towards
> > > scoping
> > > a Sidecar release. These are the items that were called out that we
> will
> > > need
> > > for a release:
> > > - Documentation
> > > - Authorization / Authentication
> > > - vnode support
> > >
> > > There are some smaller bug fixes that need to be included that we can
> label
> > > as part of the 1.0 release.[1][2][3]
> > >
> > > If there are any other tasks we need to completed, I encourage the
> > > community
> > > to create JIRAs that can be in the release milestone for the Sidecar.
> > >
> > > I agree with Stefan that OpenAPI is desirable. OpenAPI is something
> I've
> > > been
> > > looking into as well. I'm also glad to see Abhijeet can help with the
> > > documentation effort, I can also help on that front.
> > >
> > > Hopefully, I've captured your comments as truly as possible.
> > >
> > > Thanks again for all the feedback.
> > >
> 

Re: Status of CEP-1

[Štefan Miklošovič]

Totally agree with Jon here basically on all fronts. Apache Cassandra
Sidecar was always a hard nut to crack for me, that is probably why I have
not been involved with that a lot even that is a great tool to have and be
invested in as I was writing my own sidecar and I found a lot of
similarities and problems Apache's sidecar tries to fix. There was some
invisible barrier I have never managed to jump over. I was looking around
and I am very sorry if I just have not found it yet but there is not a list
of endpoints a sidecar has, is there? In readme and dev docs there is just
nothing. Taking it at a face value I just don't know what Sidecar is
capable of and how to use it. I see in the commit history there is a bunch
of commits mentioning S3 but it is a total blackbox for me as a potential
user.

On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad  wrote:

> I don't think we should release sidecar 1.0 without any docs.
>
> I took a look through the closed JIRAs to see what's there.  Here's what I
> found, please correct me if there's more:
>
> - Lots of stuff related to analytics.
>
> I would be pretty excited for this, but the analytics library only works
> with single token clusters.  Most folks don't run Cassandra this way.  I
> realize there's some element of everyone needs to scratch their own itch,
> but I don't think we can really call this a useful feature if the
> overwhelming majority of folks can't use it.  I've worked with a couple
> hundred teams over the years and can only think of 1 org outside of Apple
> and Netflix that used 1 token, and It was a cluster that predated v-nodes.
>
> The analytics repo says it's compatible with Cassandra 4, but not 5.
>
> - Backup & Restore from S3
>
> Is this compatible with other cloud providers or object stores?  It
> specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
> correct in reading it supports backing up snapshots, no continuous
> backups?  Seems like we should have at least feature parity with Medusa if
> we're going to release something here.
>
> All the other closed JIRAs look related to these two items.  So the
> question is, are we releasing 1.0 as an limited S3 backup and restore
> tool?  One that prevents you from upgrading to Cassandra 5 if you happen to
> use single token clusters?
>
> Who is the target audience?
>
> Jon
>
>
>
> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi  wrote:
>
>> Currently the Sidecar has a lot of functionality that is immediately
>> usable by the community. Apart from minor fixes, the AuthN/Z story would be
>> wrapped up soon. Post this, I would propose moving forward with cutting a
>> release with the existing feature set so we can get this in the hands of
>> our community.
>>
>> On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell  wrote:
>>
>>> Have the same question ： what ‘s the plan ？
>>>
>>> Jeff Jirsa 于2024年10月2日 周三上午10:43写道：
>>>


 On Oct 1, 2024, at 7:26 PM, Josh McKenzie  wrote:

 However it is used by a number of other features as a dependency such
 as analytics, backup/restore, repair, metrics, and CDC

 It seems like a natural pressure relief valve for moving operations out
 of a core C* node that are well served out of process.


 Yea, but the point of the foundation is to RELEASE software for the
 public good, and the link asserting consensus was dec2018, so its’ 5.5
 years and no releases.

 What’s the plan here?

Re: [DISCUSS] Modeling JIRA fix version for subprojects

[Josh McKenzie]

> - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as in 
> any subproject channels
> - some cadence of dev@ ML updates, e.g. on activities, or dependency changes, 
> etc
> - regular releases
Agree on all 3 points. Also - I've *definitely* fallen off on the project 
updates for mainline; I'll pick that back up after ApacheCon.


On Wed, Oct 2, 2024, at 1:57 AM, Mick Semb Wever wrote:
> To play devil's advocate here, it's important that the subprojects don't lose 
> visibility and silo from the rest of the project.
> 
> There are different ways to solve this, and lumping everything into one jira 
> project is a messy and poor way of doing it.  But as the sidecar has shown 
> us, subproject activity should somehow be made noisy to us.  We need sorts of 
> common spaces in the project.
> 
> If we go the separate jira project route, then some suggestions to help with 
> this are:
> - Qbot notifications in #cassandra-dev and #cassandra-noise , as well as in 
> any subproject channels
> - some cadence of dev@ ML updates, e.g. on activities, or dependency changes, 
> etc
> - regular releases
> 
> 
> On Tue, 9 Apr 2024 at 04:11, Dinesh Joshi  wrote:
>> hi folks - sorry to have dropped the ball on responding to this thread.
>> 
>> My 2 cents are as follows - 
>> 
>> 1. Having a separate JIRA project for each sub-project will add management 
>> overhead. This option, however, allows us to model unique workflows for the 
>> sub-project.
>> 
>> 2. Managing the sub-project as part of the Cassandra JIRA project would 
>> imply less management overhead but the sub-project would need to conform to 
>> the same workflows.
>> 
>> I would pick option 1 unless there is a strong reason and desire to manage a 
>> separate Jira project. We can always split out the Java Driver project if 
>> things don't work out. OTOH merging a Jira project is harder.
>> 
>> Thanks,
>> 
>> Dinesh
>> 
>> On Thu, Apr 4, 2024 at 12:45 PM Abe Ratnofsky  wrote:
>>> CEP-8 proposes using separate Jira projects per Cassandra sub-project:
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+DataStax+Drivers+Donation
>>> 
>>> > We suggest distinct Jira projects, one per driver, all to be created.
>>> 
>>> I don't see any discussion changing that from the [DISCUSS] or vote threads:
>>> https://lists.apache.org/thread/01pljcncyjyo467l5orh8nf9okrh7oxm
>>> https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>>> https://lists.apache.org/thread/crolkrhd4y6tt3k4hsy204xomshlcp4p
>>> 
>>> But looks like upon acceptance that was changed:
>>> https://lists.apache.org/thread/dhov01s8dvvh3882oxhkmmfv4tqdd68o
>>> 
>>> > New issues will be tracked under the CASSANDRA project on Apache’s JIRA 
>>> >  under the component 
>>> > ‘Client/java-driver’.
>>> 
>>> I'm in favor of using the same Jira as Cassandra proper. Committership is 
>>> project-wide, so having a standardized process (same ticket flow, review 
>>> rules, labels, etc. is beneficial). But multiple votes happened based on 
>>> the content of the CEP, so we should stick to what was voted on and move to 
>>> a separate Jira.
>>> 
>>> --
>>> Abe

Re: Status of CEP-1

[Štefan Miklošovič]

Something like this:

https://instaclustr.github.io/instaclustr-icarus-go-client/

On Wed, Oct 2, 2024 at 4:54 PM Štefan Miklošovič 
wrote:

> While documenting endpoints please use something like OpenAPI
> specification. The sidecar should expose this document itself so when I go
> to this and that URL, I see all endpoints, I put the payloads / parameters
> for them and I can just directly call that, no messing with curl / wget or
> programmatically or whatever like that. The barrier to exercise the basic
> functionality has to virtually not be there.
>
> On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey 
> wrote:
>
>> Hi folks,
>>
>> I have been using Sidecar recently and have found some of its
>> functionalities to be quite useful. Hari and I are also working on CEP-40
>> which aims to introduce live migration features in Sidecar in the
>> near future.
>>
>> However, as others have mentioned, I agree that it currently lacks proper
>> documentation.
>>
>> Since this is an official Apache project, I believe that creating
>> comprehensive documentation would be beneficial. This documentation should
>> include an overview, architecture, a list and description of various
>> endpoints, and some examples or tutorials on how to use Sidecar's features.
>>
>> This documentation would help people get started with Sidecar and lower
>> the entry barrier for many. We can update the documentation incrementally
>> as needed, along with future enhancements and new features. However,
>> creating some form of formal documentation would be very helpful.
>>
>> To this end I'm willing and highly interested in writing some form of
>> formal documentation for the Sidecar project. Please let me know your
>> thoughts/opinions on this proposal.
>>
>>
>>
>> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič 
>> wrote:
>>
>>> Totally agree with Jon here basically on all fronts. Apache Cassandra
>>> Sidecar was always a hard nut to crack for me, that is probably why I have
>>> not been involved with that a lot even that is a great tool to have and be
>>> invested in as I was writing my own sidecar and I found a lot of
>>> similarities and problems Apache's sidecar tries to fix. There was some
>>> invisible barrier I have never managed to jump over. I was looking around
>>> and I am very sorry if I just have not found it yet but there is not a list
>>> of endpoints a sidecar has, is there? In readme and dev docs there is just
>>> nothing. Taking it at a face value I just don't know what Sidecar is
>>> capable of and how to use it. I see in the commit history there is a bunch
>>> of commits mentioning S3 but it is a total blackbox for me as a potential
>>> user.
>>>
>>> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad 
>>> wrote:
>>>
 I don't think we should release sidecar 1.0 without any docs.

 I took a look through the closed JIRAs to see what's there.  Here's
 what I found, please correct me if there's more:

 - Lots of stuff related to analytics.

 I would be pretty excited for this, but the analytics library only
 works with single token clusters.  Most folks don't run Cassandra this
 way.  I realize there's some element of everyone needs to scratch their own
 itch, but I don't think we can really call this a useful feature if the
 overwhelming majority of folks can't use it.  I've worked with a couple
 hundred teams over the years and can only think of 1 org outside of Apple
 and Netflix that used 1 token, and It was a cluster that predated v-nodes.

 The analytics repo says it's compatible with Cassandra 4, but not 5.

 - Backup & Restore from S3

 Is this compatible with other cloud providers or object stores?  It
 specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
 correct in reading it supports backing up snapshots, no continuous
 backups?  Seems like we should have at least feature parity with Medusa if
 we're going to release something here.

 All the other closed JIRAs look related to these two items.  So the
 question is, are we releasing 1.0 as an limited S3 backup and restore
 tool?  One that prevents you from upgrading to Cassandra 5 if you happen to
 use single token clusters?

 Who is the target audience?

 Jon



 On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi  wrote:

> Currently the Sidecar has a lot of functionality that is immediately
> usable by the community. Apart from minor fixes, the AuthN/Z story would 
> be
> wrapped up soon. Post this, I would propose moving forward with cutting a
> release with the existing feature set so we can get this in the hands of
> our community.
>
> On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell 
> wrote:
>
>> Have the same question ： what ‘s the plan ？
>>
>> Jeff Jirsa 于2024年10月2日 周三上午10:43写道：
>>
>>>
>>>
>>> On Oct 1, 2024, at 7:26 PM, Josh McKenzie 
>

Re: Status of CEP-1

[Jon Haddad]

Hi Abhijeet,

Contributing documentation would be greatly appreciated.

Feature wise - is there functionality in sidecar that I missed?  What are
you finding to be quite useful?  I think everyone would benefit to hear
from folks who use it and are getting something useful out of it as it will
help frame the discussion.

Jon

On Wed, Oct 2, 2024 at 10:13 AM Abhijeet Dubey 
wrote:

> Hi folks,
>
> I have been using Sidecar recently and have found some of its
> functionalities to be quite useful. Hari and I are also working on CEP-40
> which aims to introduce live migration features in Sidecar in the
> near future.
>
> However, as others have mentioned, I agree that it currently lacks proper
> documentation.
>
> Since this is an official Apache project, I believe that creating
> comprehensive documentation would be beneficial. This documentation should
> include an overview, architecture, a list and description of various
> endpoints, and some examples or tutorials on how to use Sidecar's features.
>
> This documentation would help people get started with Sidecar and lower
> the entry barrier for many. We can update the documentation incrementally
> as needed, along with future enhancements and new features. However,
> creating some form of formal documentation would be very helpful.
>
> To this end I'm willing and highly interested in writing some form of
> formal documentation for the Sidecar project. Please let me know your
> thoughts/opinions on this proposal.
>
>
>
> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič 
> wrote:
>
>> Totally agree with Jon here basically on all fronts. Apache Cassandra
>> Sidecar was always a hard nut to crack for me, that is probably why I have
>> not been involved with that a lot even that is a great tool to have and be
>> invested in as I was writing my own sidecar and I found a lot of
>> similarities and problems Apache's sidecar tries to fix. There was some
>> invisible barrier I have never managed to jump over. I was looking around
>> and I am very sorry if I just have not found it yet but there is not a list
>> of endpoints a sidecar has, is there? In readme and dev docs there is just
>> nothing. Taking it at a face value I just don't know what Sidecar is
>> capable of and how to use it. I see in the commit history there is a bunch
>> of commits mentioning S3 but it is a total blackbox for me as a potential
>> user.
>>
>> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad 
>> wrote:
>>
>>> I don't think we should release sidecar 1.0 without any docs.
>>>
>>> I took a look through the closed JIRAs to see what's there.  Here's what
>>> I found, please correct me if there's more:
>>>
>>> - Lots of stuff related to analytics.
>>>
>>> I would be pretty excited for this, but the analytics library only works
>>> with single token clusters.  Most folks don't run Cassandra this way.  I
>>> realize there's some element of everyone needs to scratch their own itch,
>>> but I don't think we can really call this a useful feature if the
>>> overwhelming majority of folks can't use it.  I've worked with a couple
>>> hundred teams over the years and can only think of 1 org outside of Apple
>>> and Netflix that used 1 token, and It was a cluster that predated v-nodes.
>>>
>>> The analytics repo says it's compatible with Cassandra 4, but not 5.
>>>
>>> - Backup & Restore from S3
>>>
>>> Is this compatible with other cloud providers or object stores?  It
>>> specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
>>> correct in reading it supports backing up snapshots, no continuous
>>> backups?  Seems like we should have at least feature parity with Medusa if
>>> we're going to release something here.
>>>
>>> All the other closed JIRAs look related to these two items.  So the
>>> question is, are we releasing 1.0 as an limited S3 backup and restore
>>> tool?  One that prevents you from upgrading to Cassandra 5 if you happen to
>>> use single token clusters?
>>>
>>> Who is the target audience?
>>>
>>> Jon
>>>
>>>
>>>
>>> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi  wrote:
>>>
 Currently the Sidecar has a lot of functionality that is immediately
 usable by the community. Apart from minor fixes, the AuthN/Z story would be
 wrapped up soon. Post this, I would propose moving forward with cutting a
 release with the existing feature set so we can get this in the hands of
 our community.

 On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell 
 wrote:

> Have the same question ： what ‘s the plan ？
>
> Jeff Jirsa 于2024年10月2日 周三上午10:43写道：
>
>>
>>
>> On Oct 1, 2024, at 7:26 PM, Josh McKenzie 
>> wrote:
>>
>> However it is used by a number of other features as a dependency such
>> as analytics, backup/restore, repair, metrics, and CDC
>>
>> It seems like a natural pressure relief valve for moving operations
>> out of a core C* node that are well served out of process.
>>
>>
>> Yea, but the point o

Re: Status of CEP-1

[Štefan Miklošovič]

While documenting endpoints please use something like OpenAPI
specification. The sidecar should expose this document itself so when I go
to this and that URL, I see all endpoints, I put the payloads / parameters
for them and I can just directly call that, no messing with curl / wget or
programmatically or whatever like that. The barrier to exercise the basic
functionality has to virtually not be there.

On Wed, Oct 2, 2024 at 4:13 PM Abhijeet Dubey 
wrote:

> Hi folks,
>
> I have been using Sidecar recently and have found some of its
> functionalities to be quite useful. Hari and I are also working on CEP-40
> which aims to introduce live migration features in Sidecar in the
> near future.
>
> However, as others have mentioned, I agree that it currently lacks proper
> documentation.
>
> Since this is an official Apache project, I believe that creating
> comprehensive documentation would be beneficial. This documentation should
> include an overview, architecture, a list and description of various
> endpoints, and some examples or tutorials on how to use Sidecar's features.
>
> This documentation would help people get started with Sidecar and lower
> the entry barrier for many. We can update the documentation incrementally
> as needed, along with future enhancements and new features. However,
> creating some form of formal documentation would be very helpful.
>
> To this end I'm willing and highly interested in writing some form of
> formal documentation for the Sidecar project. Please let me know your
> thoughts/opinions on this proposal.
>
>
>
> On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič 
> wrote:
>
>> Totally agree with Jon here basically on all fronts. Apache Cassandra
>> Sidecar was always a hard nut to crack for me, that is probably why I have
>> not been involved with that a lot even that is a great tool to have and be
>> invested in as I was writing my own sidecar and I found a lot of
>> similarities and problems Apache's sidecar tries to fix. There was some
>> invisible barrier I have never managed to jump over. I was looking around
>> and I am very sorry if I just have not found it yet but there is not a list
>> of endpoints a sidecar has, is there? In readme and dev docs there is just
>> nothing. Taking it at a face value I just don't know what Sidecar is
>> capable of and how to use it. I see in the commit history there is a bunch
>> of commits mentioning S3 but it is a total blackbox for me as a potential
>> user.
>>
>> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad 
>> wrote:
>>
>>> I don't think we should release sidecar 1.0 without any docs.
>>>
>>> I took a look through the closed JIRAs to see what's there.  Here's what
>>> I found, please correct me if there's more:
>>>
>>> - Lots of stuff related to analytics.
>>>
>>> I would be pretty excited for this, but the analytics library only works
>>> with single token clusters.  Most folks don't run Cassandra this way.  I
>>> realize there's some element of everyone needs to scratch their own itch,
>>> but I don't think we can really call this a useful feature if the
>>> overwhelming majority of folks can't use it.  I've worked with a couple
>>> hundred teams over the years and can only think of 1 org outside of Apple
>>> and Netflix that used 1 token, and It was a cluster that predated v-nodes.
>>>
>>> The analytics repo says it's compatible with Cassandra 4, but not 5.
>>>
>>> - Backup & Restore from S3
>>>
>>> Is this compatible with other cloud providers or object stores?  It
>>> specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
>>> correct in reading it supports backing up snapshots, no continuous
>>> backups?  Seems like we should have at least feature parity with Medusa if
>>> we're going to release something here.
>>>
>>> All the other closed JIRAs look related to these two items.  So the
>>> question is, are we releasing 1.0 as an limited S3 backup and restore
>>> tool?  One that prevents you from upgrading to Cassandra 5 if you happen to
>>> use single token clusters?
>>>
>>> Who is the target audience?
>>>
>>> Jon
>>>
>>>
>>>
>>> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi  wrote:
>>>
 Currently the Sidecar has a lot of functionality that is immediately
 usable by the community. Apart from minor fixes, the AuthN/Z story would be
 wrapped up soon. Post this, I would propose moving forward with cutting a
 release with the existing feature set so we can get this in the hands of
 our community.

 On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell 
 wrote:

> Have the same question ： what ‘s the plan ？
>
> Jeff Jirsa 于2024年10月2日 周三上午10:43写道：
>
>>
>>
>> On Oct 1, 2024, at 7:26 PM, Josh McKenzie 
>> wrote:
>>
>> However it is used by a number of other features as a dependency such
>> as analytics, backup/restore, repair, metrics, and CDC
>>
>> It seems like a natural pressure relief valve for moving operations
>> out of a core C* node

Re: Status of CEP-1

[Abhijeet Dubey]

Hi folks,

I have been using Sidecar recently and have found some of its
functionalities to be quite useful. Hari and I are also working on CEP-40
which aims to introduce live migration features in Sidecar in the
near future.

However, as others have mentioned, I agree that it currently lacks proper
documentation.

Since this is an official Apache project, I believe that creating
comprehensive documentation would be beneficial. This documentation should
include an overview, architecture, a list and description of various
endpoints, and some examples or tutorials on how to use Sidecar's features.

This documentation would help people get started with Sidecar and lower the
entry barrier for many. We can update the documentation incrementally as
needed, along with future enhancements and new features. However, creating
some form of formal documentation would be very helpful.

To this end I'm willing and highly interested in writing some form of
formal documentation for the Sidecar project. Please let me know your
thoughts/opinions on this proposal.



On Wed, Oct 2, 2024 at 6:46 PM Štefan Miklošovič 
wrote:

> Totally agree with Jon here basically on all fronts. Apache Cassandra
> Sidecar was always a hard nut to crack for me, that is probably why I have
> not been involved with that a lot even that is a great tool to have and be
> invested in as I was writing my own sidecar and I found a lot of
> similarities and problems Apache's sidecar tries to fix. There was some
> invisible barrier I have never managed to jump over. I was looking around
> and I am very sorry if I just have not found it yet but there is not a list
> of endpoints a sidecar has, is there? In readme and dev docs there is just
> nothing. Taking it at a face value I just don't know what Sidecar is
> capable of and how to use it. I see in the commit history there is a bunch
> of commits mentioning S3 but it is a total blackbox for me as a potential
> user.
>
> On Wed, Oct 2, 2024 at 2:52 PM Jon Haddad  wrote:
>
>> I don't think we should release sidecar 1.0 without any docs.
>>
>> I took a look through the closed JIRAs to see what's there.  Here's what
>> I found, please correct me if there's more:
>>
>> - Lots of stuff related to analytics.
>>
>> I would be pretty excited for this, but the analytics library only works
>> with single token clusters.  Most folks don't run Cassandra this way.  I
>> realize there's some element of everyone needs to scratch their own itch,
>> but I don't think we can really call this a useful feature if the
>> overwhelming majority of folks can't use it.  I've worked with a couple
>> hundred teams over the years and can only think of 1 org outside of Apple
>> and Netflix that used 1 token, and It was a cluster that predated v-nodes.
>>
>> The analytics repo says it's compatible with Cassandra 4, but not 5.
>>
>> - Backup & Restore from S3
>>
>> Is this compatible with other cloud providers or object stores?  It
>> specifically lists S3 in JIRA.  I haven't looked at the source yet.  Am I
>> correct in reading it supports backing up snapshots, no continuous
>> backups?  Seems like we should have at least feature parity with Medusa if
>> we're going to release something here.
>>
>> All the other closed JIRAs look related to these two items.  So the
>> question is, are we releasing 1.0 as an limited S3 backup and restore
>> tool?  One that prevents you from upgrading to Cassandra 5 if you happen to
>> use single token clusters?
>>
>> Who is the target audience?
>>
>> Jon
>>
>>
>>
>> On Wed, Oct 2, 2024 at 2:41 AM Dinesh Joshi  wrote:
>>
>>> Currently the Sidecar has a lot of functionality that is immediately
>>> usable by the community. Apart from minor fixes, the AuthN/Z story would be
>>> wrapped up soon. Post this, I would propose moving forward with cutting a
>>> release with the existing feature set so we can get this in the hands of
>>> our community.
>>>
>>> On Tue, Oct 1, 2024 at 8:27 PM guo Maxwell  wrote:
>>>
 Have the same question ： what ‘s the plan ？

 Jeff Jirsa 于2024年10月2日 周三上午10:43写道：

>
>
> On Oct 1, 2024, at 7:26 PM, Josh McKenzie 
> wrote:
>
> However it is used by a number of other features as a dependency such
> as analytics, backup/restore, repair, metrics, and CDC
>
> It seems like a natural pressure relief valve for moving operations
> out of a core C* node that are well served out of process.
>
>
> Yea, but the point of the foundation is to RELEASE software for the
> public good, and the link asserting consensus was dec2018, so its’ 5.5
> years and no releases.
>
> What’s the plan here?
>
>
>
>
>

-- 
*Abhijeet*