Re: [VOTE] CEP-3: Guardrails

[Andrés de la Peña]

The vote passes with 12 +1 votes and no -1 votes.

I'll shortly open some tickets to start implementing guardrails.

On Fri, 12 Nov 2021 at 14:51, Joshua McKenzie  wrote:

> +1
>
> On Thu, Nov 11, 2021 at 12:06 PM David Capwell  >
> wrote:
>
> > +1
> >
> > > On Nov 11, 2021, at 7:10 AM, Sumanth Pasupuleti <
> > sumanth.pasupuleti...@gmail.com> wrote:
> > >
> > > +1
> > >
> > > On Thu, Nov 11, 2021 at 6:10 AM Gary Dusbabek 
> > wrote:
> > >
> > >> +1
> > >>
> > >> On Thu, Nov 11, 2021 at 5:30 AM Andrés de la Peña <
> adelap...@apache.org
> > >
> > >> wrote:
> > >>
> > >>> Hi everyone,
> > >>>
> > >>> I would like to start a vote on this CEP.
> > >>>
> > >>> Proposal:
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails
> > >>>
> > >>> Discussion:
> > >>> https://lists.apache.org/thread/7f6lntfdnkpqr7o0h2d2jlg8q7gf54w2
> > >>> https://lists.apache.org/thread/0bd6fo4hdnwc8q2sq4xwvv4nqpxw10ds
> > >>>
> > >>> The vote will be open for 72 hours.
> > >>> A vote passes if there are at least three binding +1s and no binding
> > >>> vetoes.
> > >>>
> > >>> Thanks,
> > >>>
> > >>
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

[Jacek Lewandowski]

I'd put it another way - the scope is to make it possible to provide a new
implementation of sstable format without the necessity to refactor
Cassandra code. It implies a contract about the responsibilities of sstable
format implementation so that the rest of the code can rely on that, and
only on that, and do not make assumptions beyond that. But it does not
claim that the created interfaces will not change even with a minor version
release. When those interfaces are around for sometime, we can start a
separate discussion about whether we want to put some guarantees on them.

- - -- --- -  -
Jacek Lewandowski


On Wed, Nov 10, 2021 at 9:01 PM David Capwell 
wrote:

> If this gets descoped to test only (can break all interfaces in a minor)
> then my support concerns are no longer valid; I am cool with the CEP scoped
> only to improving testing
>
> > On Nov 10, 2021, at 11:20 AM, Jacek Lewandowski <
> lewandowski.ja...@gmail.com> wrote:
> >
> > For the other ticket (schema update handler interface) I was also
> proposing
> > a kind of @DeveloperApi annotation as seen in other projects but
> similarly
> > to this thread there were different opinions and no conclusion. After
> > reading the comments I must agree that perhaps it is way too early to
> mark
> > this interface as stable. Perhaps it was too far-fetched to say it would
> be
> > for people who wished to replace the SSTable format. My focus is
> > primarily on cleaning up the code (modularization and clean contracts)
> and
> > making it possible to introduce a new format in the future while allowing
> > us to maintain the old format (no "if then else" approach)
> >
> > - - -- --- -  -
> > Jacek Lewandowski
> >
> >
> > On Wed, Nov 10, 2021 at 12:53 AM bened...@apache.org <
> bened...@apache.org>
> > wrote:
> >
> >>> I may be wrong here, but the CEP directly calls out making this api
> >> public for people who wish to replace the SSTable format
> >>
> >> I don’t think this implies API stability. For starters, it doesn’t
> >> stipulate that these implementations will be supported out of tree (the
> >> only one I’m aware of, so far as I understand, is intended to be
> incubated
> >> in tree), nor does an API for external usage have to be stable. It’s
> fine
> >> to create an API and tell users it’s unstable, and that they should
> closely
> >> monitor patch version changes if they use it.
> >>
> >> That said, norms may be changing around what can go into patch releases
> >> anyhow, so this may be a lot of noise about nothing. If all new
> development
> >> goes into trunk, then it’s all moot. But I don’t think we can make hard
> >> assumptions about that today, as historically these sorts of intentions
> >> haven’t lasted.
> >>
> >> I’m fairly against the idea of introducing hard restrictions on this,
> and
> >> potentially ossifying the codebase. I’m not keen to even consider out of
> >> tree consumers of these APIs in any way, for compatibility,
> upgradeability
> >> or anything. There’s a lot that needs to be done over the coming years
> to
> >> improve the internal structure of the project, and unduly entrenching
> the
> >> current state of affairs would be a huge potential harm of these
> efforts to
> >> modularise the codebase.
> >>
> >> From: David Capwell 
> >> Date: Tuesday, 9 November 2021 at 23:38
> >> To: dev@cassandra.apache.org 
> >> Subject: Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)
> >>> My understanding is that the only interface that is expected to be
> >> stable for external consumers is the secondary index API
> >>
> >> I may be wrong here, but the CEP directly calls out making this api
> public
> >> for people who wish to replace the SSTable format ("Cassandra developers
> >> who want to develop and publish different file format
> implementations."),
> >> so if we need to support 2i API, why would we not support SSTable API as
> >> well?
> >>
> >>> All of the other mentioned APIs are in my opinion for internal usage
> only
> >>
> >> This gets back to my point; it is currently tribal knowledge what needs
> to
> >> work and what doesn’t, and without the broader set of committers knowing
> >> this then the likely hood any new API will break in a minor is high.
> >>
> >>> On Nov 9, 2021, at 12:13 PM, bened...@apache.org wrote:
> >>>
> >>> I agree that we don’t need to block the CEP on this, and that we should
> >> have that discussion. But it’s worth noting that the CEP should not
> >> anticipate or depend on any specific outcome of that discussion.
> >>>
> >>> Since it is somewhat relevant for this discussion, my view is that no
> >> interface should be assumed to be stable without the prior explicit
> >> agreement of the community.
> >>>
> >>> My understanding is that the only interface that is expected to be
> >> stable for external consumers is the secondary index API. Perhaps also
> >> snitches? But also perhaps not, as the difficulty of upgrading these at
> the
> >> sa

Re: [DISCUSS] Mentoring newcomers

[Jacek Lewandowski]

I'm in

- - -- --- -  -
Jacek Lewandowski


On Mon, Nov 15, 2021 at 7:34 AM Berenguer Blasi 
wrote:

> I'm in as well
>
> On 14/11/21 14:55, Joshua McKenzie wrote:
> > Sign me up.
> >
> >
> > On Fri, Nov 12, 2021 at 4:38 PM David Capwell  >
> > wrote:
> >
> >> I am cool helping.
> >>
> >>> On Nov 12, 2021, at 10:29 AM, Ekaterina Dimitrova <
> e.dimitr...@gmail.com>
> >> wrote:
> >>> I am in too
> >>>
> >>> On Fri, 12 Nov 2021 at 13:23,  wrote:
> >>>
>  I am interested as well.
> 
>  Sent from my iPhone
> 
> > On 12. Nov 2021, at 19:01, Paulo Motta 
> >> wrote:
> > Count me in.
> >
> >> Em sex., 12 de nov. de 2021 às 14:16, Brandon Williams <
>  dri...@gmail.com>
> >> escreveu:
> >>
> >> I'm interested.
> >>
> >>> On Fri, Nov 12, 2021 at 11:05 AM Benjamin Lerer  >
>  wrote:
> >>> Hi everybody
> >>>
> >>> As discussed in the *Creating a new slack channel for newcomers*
>  thead, a
> >>> solution to help newcomers engage with the project would be to
> >> provide
>  a
> >>> list of mentors that newcomers can contact when they feel insecure
>  about
> >>> asking questions through our cassandra-dev channel or through the
>  mailing
> >>> list.
> >>>
> >>> I would like to collect the list of people that are interested in
>  helping
> >>> out newcomers so that we can post that list on our website.
> >>
> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>
>  -
>  To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>  For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

[David Capwell]

Works for me

> On Nov 15, 2021, at 4:21 AM, Jacek Lewandowski  
> wrote:
> 
> I'd put it another way - the scope is to make it possible to provide a new
> implementation of sstable format without the necessity to refactor
> Cassandra code. It implies a contract about the responsibilities of sstable
> format implementation so that the rest of the code can rely on that, and
> only on that, and do not make assumptions beyond that. But it does not
> claim that the created interfaces will not change even with a minor version
> release. When those interfaces are around for sometime, we can start a
> separate discussion about whether we want to put some guarantees on them.
> 
> - - -- --- -  -
> Jacek Lewandowski
> 
> 
> On Wed, Nov 10, 2021 at 9:01 PM David Capwell 
> wrote:
> 
>> If this gets descoped to test only (can break all interfaces in a minor)
>> then my support concerns are no longer valid; I am cool with the CEP scoped
>> only to improving testing
>> 
>>> On Nov 10, 2021, at 11:20 AM, Jacek Lewandowski <
>> lewandowski.ja...@gmail.com> wrote:
>>> 
>>> For the other ticket (schema update handler interface) I was also
>> proposing
>>> a kind of @DeveloperApi annotation as seen in other projects but
>> similarly
>>> to this thread there were different opinions and no conclusion. After
>>> reading the comments I must agree that perhaps it is way too early to
>> mark
>>> this interface as stable. Perhaps it was too far-fetched to say it would
>> be
>>> for people who wished to replace the SSTable format. My focus is
>>> primarily on cleaning up the code (modularization and clean contracts)
>> and
>>> making it possible to introduce a new format in the future while allowing
>>> us to maintain the old format (no "if then else" approach)
>>> 
>>> - - -- --- -  -
>>> Jacek Lewandowski
>>> 
>>> 
>>> On Wed, Nov 10, 2021 at 12:53 AM bened...@apache.org <
>> bened...@apache.org>
>>> wrote:
>>> 
> I may be wrong here, but the CEP directly calls out making this api
 public for people who wish to replace the SSTable format
 
 I don’t think this implies API stability. For starters, it doesn’t
 stipulate that these implementations will be supported out of tree (the
 only one I’m aware of, so far as I understand, is intended to be
>> incubated
 in tree), nor does an API for external usage have to be stable. It’s
>> fine
 to create an API and tell users it’s unstable, and that they should
>> closely
 monitor patch version changes if they use it.
 
 That said, norms may be changing around what can go into patch releases
 anyhow, so this may be a lot of noise about nothing. If all new
>> development
 goes into trunk, then it’s all moot. But I don’t think we can make hard
 assumptions about that today, as historically these sorts of intentions
 haven’t lasted.
 
 I’m fairly against the idea of introducing hard restrictions on this,
>> and
 potentially ossifying the codebase. I’m not keen to even consider out of
 tree consumers of these APIs in any way, for compatibility,
>> upgradeability
 or anything. There’s a lot that needs to be done over the coming years
>> to
 improve the internal structure of the project, and unduly entrenching
>> the
 current state of affairs would be a huge potential harm of these
>> efforts to
 modularise the codebase.
 
 From: David Capwell 
 Date: Tuesday, 9 November 2021 at 23:38
 To: dev@cassandra.apache.org 
 Subject: Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)
> My understanding is that the only interface that is expected to be
 stable for external consumers is the secondary index API
 
 I may be wrong here, but the CEP directly calls out making this api
>> public
 for people who wish to replace the SSTable format ("Cassandra developers
 who want to develop and publish different file format
>> implementations."),
 so if we need to support 2i API, why would we not support SSTable API as
 well?
 
> All of the other mentioned APIs are in my opinion for internal usage
>> only
 
 This gets back to my point; it is currently tribal knowledge what needs
>> to
 work and what doesn’t, and without the broader set of committers knowing
 this then the likely hood any new API will break in a minor is high.
 
> On Nov 9, 2021, at 12:13 PM, bened...@apache.org wrote:
> 
> I agree that we don’t need to block the CEP on this, and that we should
 have that discussion. But it’s worth noting that the CEP should not
 anticipate or depend on any specific outcome of that discussion.
> 
> Since it is somewhat relevant for this discussion, my view is that no
 interface should be assumed to be stable without the prior explicit
 agreement of the community.
> 
> My understanding is that the only interface that is expected to be
 stable for 

Re: Resurrection of CASSANDRA-9633 - SSTable encryption

[Jeremiah D Jordan]

> On Nov 14, 2021, at 3:53 PM, Stefan Miklosovic 
>  wrote:
> 
> Hey,
> 
> there are two points we are not completely sure about.
> 
> The first one is streaming. If there is a cluster of 5 nodes, each
> node has its own unique encryption key. Hence, if a SSTable is stored
> on a disk with the key for node 1 and this is streamed to node 2 -
> which has a different key - it would not be able to decrypt that. Our
> idea is to actually send data over the wire _decrypted_ however it
> would be still secure if internode communication is done via TLS. Is
> this approach good with you?
> 

So would you fail startup if someone enabled sstable encryption but did not 
have TLS for internode communication?  Another concern here is making sure zero 
copy streaming does not get triggered for this case.
Have you considered having some way to distribute the keys to all nodes such 
that you don’t need to decrypt on the sending side?  Having to do this will 
mean a lot more overhead for the sending side of a streaming operation.

> The second question is about key rotation. If an operator needs to
> roll the key because it was compromised or there is some policy around
> that, we should be able to provide some way to rotate it. Our idea is
> to write a tool (either a subcommand of nodetool (rewritesstables)
> command or a completely standalone one in tools) which would take the
> first, original key, the second, new key and dir with sstables as
> input and it would literally took the data and it would rewrite it to
> the second set of sstables which would be encrypted with the second
> key. What do you think about this?

I would rather suggest that “what key encrypted this” be part of the sstable 
metadata, and allow there to be multiple keys in the system.  This way you can 
just add a new “current key” so new sstables use the new key, but existing 
sstables would use the old key.  An operator could then trigger a “nodetool 
upgradesstables —all” to rewrite the existing sstables with the new “current 
key”.

> 
> Regards
> 
> On Sat, 13 Nov 2021 at 19:35,  wrote:
>> 
>> Same reaction here - great to have traction on this ticket. Shylaja, thanks 
>> for your work on this and to Stefan as well! It would be wonderful to have 
>> the feature complete.
>> 
>> One thing I’d mention is that a lot’s changed about the project’s testing 
>> strategy since the original patch was written. I see that the 2016 version 
>> adds a couple round-trip unit tests with a small amount of static data. It 
>> would be good to see randomized tests fleshed out that exercise more of the 
>> read/write path; or which add variants of existing read/write path tests 
>> that enable encryption.
>> 
>> – Scott
>> 
>>> On Nov 13, 2021, at 7:53 AM, Brandon Williams  wrote:
>>> 
>>> We already have a ticket and this predated CEPs, and being an
>>> obviously good improvement to have that many have been asking for for
>>> some time now, I don't see the need for a CEP here.
>>> 
>>> On Sat, Nov 13, 2021 at 5:01 AM Stefan Miklosovic
>>>  wrote:
 
 Hi list,
 
 an engineer from Intel - Shylaja Kokoori (who is watching this list
 closely) has retrofitted the original code from CASSANDRA-9633 work in
 times of 3.4 to the current trunk with my help here and there, mostly
 cosmetic.
 
 I would like to know if there is a general consensus about me going to
 create a CEP for this feature or what is your perception on this. I
 know we have it a little bit backwards here as we should first discuss
 and then code but I am super glad that we have some POC we can
 elaborate further on and CEP would just cement  and summarise the
 approach / other implementation aspects of this feature.
 
 I think that having 9633 merged will fill quite a big operational gap
 when it comes to security. There are a lot of enterprises who desire
 this feature so much. I can not remember when I last saw a ticket with
 50 watchers which was inactive for such a long time.
 
 Regards
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
 For additional commands, e-mail: dev-h...@cassandra.apache.org
 
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, 

Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)

[Branimir Lambov]

Looks like the discussion is settled down. I am moving forward to putting
this proposal to a vote.

Regards,
Branimir

On Mon, Nov 15, 2021 at 7:28 PM David Capwell 
wrote:

> Works for me
>
> > On Nov 15, 2021, at 4:21 AM, Jacek Lewandowski <
> lewandowski.ja...@gmail.com> wrote:
> >
> > I'd put it another way - the scope is to make it possible to provide a
> new
> > implementation of sstable format without the necessity to refactor
> > Cassandra code. It implies a contract about the responsibilities of
> sstable
> > format implementation so that the rest of the code can rely on that, and
> > only on that, and do not make assumptions beyond that. But it does not
> > claim that the created interfaces will not change even with a minor
> version
> > release. When those interfaces are around for sometime, we can start a
> > separate discussion about whether we want to put some guarantees on them.
> >
> > - - -- --- -  -
> > Jacek Lewandowski
> >
> >
> > On Wed, Nov 10, 2021 at 9:01 PM David Capwell  >
> > wrote:
> >
> >> If this gets descoped to test only (can break all interfaces in a minor)
> >> then my support concerns are no longer valid; I am cool with the CEP
> scoped
> >> only to improving testing
> >>
> >>> On Nov 10, 2021, at 11:20 AM, Jacek Lewandowski <
> >> lewandowski.ja...@gmail.com> wrote:
> >>>
> >>> For the other ticket (schema update handler interface) I was also
> >> proposing
> >>> a kind of @DeveloperApi annotation as seen in other projects but
> >> similarly
> >>> to this thread there were different opinions and no conclusion. After
> >>> reading the comments I must agree that perhaps it is way too early to
> >> mark
> >>> this interface as stable. Perhaps it was too far-fetched to say it
> would
> >> be
> >>> for people who wished to replace the SSTable format. My focus is
> >>> primarily on cleaning up the code (modularization and clean contracts)
> >> and
> >>> making it possible to introduce a new format in the future while
> allowing
> >>> us to maintain the old format (no "if then else" approach)
> >>>
> >>> - - -- --- -  -
> >>> Jacek Lewandowski
> >>>
> >>>
> >>> On Wed, Nov 10, 2021 at 12:53 AM bened...@apache.org <
> >> bened...@apache.org>
> >>> wrote:
> >>>
> > I may be wrong here, but the CEP directly calls out making this api
>  public for people who wish to replace the SSTable format
> 
>  I don’t think this implies API stability. For starters, it doesn’t
>  stipulate that these implementations will be supported out of tree
> (the
>  only one I’m aware of, so far as I understand, is intended to be
> >> incubated
>  in tree), nor does an API for external usage have to be stable. It’s
> >> fine
>  to create an API and tell users it’s unstable, and that they should
> >> closely
>  monitor patch version changes if they use it.
> 
>  That said, norms may be changing around what can go into patch
> releases
>  anyhow, so this may be a lot of noise about nothing. If all new
> >> development
>  goes into trunk, then it’s all moot. But I don’t think we can make
> hard
>  assumptions about that today, as historically these sorts of
> intentions
>  haven’t lasted.
> 
>  I’m fairly against the idea of introducing hard restrictions on this,
> >> and
>  potentially ossifying the codebase. I’m not keen to even consider out
> of
>  tree consumers of these APIs in any way, for compatibility,
> >> upgradeability
>  or anything. There’s a lot that needs to be done over the coming years
> >> to
>  improve the internal structure of the project, and unduly entrenching
> >> the
>  current state of affairs would be a huge potential harm of these
> >> efforts to
>  modularise the codebase.
> 
>  From: David Capwell 
>  Date: Tuesday, 9 November 2021 at 23:38
>  To: dev@cassandra.apache.org 
>  Subject: Re: [DISCUSS] CEP-17: SSTable format API (CASSANDRA-17056)
> > My understanding is that the only interface that is expected to be
>  stable for external consumers is the secondary index API
> 
>  I may be wrong here, but the CEP directly calls out making this api
> >> public
>  for people who wish to replace the SSTable format ("Cassandra
> developers
>  who want to develop and publish different file format
> >> implementations."),
>  so if we need to support 2i API, why would we not support SSTable API
> as
>  well?
> 
> > All of the other mentioned APIs are in my opinion for internal usage
> >> only
> 
>  This gets back to my point; it is currently tribal knowledge what
> needs
> >> to
>  work and what doesn’t, and without the broader set of committers
> knowing
>  this then the likely hood any new API will break in a minor is high.
> 
> > On Nov 9, 2021, at 12:13 PM, bened...@apache.org wrote:
> >
> > I agree that we don’t need to block the CEP on this, and that we
> s

[VOTE] CEP-17: SSTable format API

[Branimir Lambov]

Hi everyone,

I would like to start a vote on this CEP.

Proposal:
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API

Discussion:
https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E

The vote will be open for 72 hours.
A vote passes if there are at least three binding +1s and no binding vetoes.

Regards,
Branimir

Re: [VOTE] CEP-17: SSTable format API

[Brandon Williams]

+1

On Mon, Nov 15, 2021 at 1:43 PM Branimir Lambov  wrote:
>
> Hi everyone,
>
> I would like to start a vote on this CEP.
>
> Proposal:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
>
> Discussion:
> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
>
> The vote will be open for 72 hours.
> A vote passes if there are at least three binding +1s and no binding vetoes.
>
> Regards,
> Branimir

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Resurrection of CASSANDRA-9633 - SSTable encryption

[Stefan Miklosovic]

On Mon, 15 Nov 2021 at 19:42, Jeremiah D Jordan
 wrote:
>
>
>
> > On Nov 14, 2021, at 3:53 PM, Stefan Miklosovic 
> >  wrote:
> >
> > Hey,
> >
> > there are two points we are not completely sure about.
> >
> > The first one is streaming. If there is a cluster of 5 nodes, each
> > node has its own unique encryption key. Hence, if a SSTable is stored
> > on a disk with the key for node 1 and this is streamed to node 2 -
> > which has a different key - it would not be able to decrypt that. Our
> > idea is to actually send data over the wire _decrypted_ however it
> > would be still secure if internode communication is done via TLS. Is
> > this approach good with you?
> >
>
> So would you fail startup if someone enabled sstable encryption but did not 
> have TLS for internode communication?  Another concern here is making sure 
> zero copy streaming does not get triggered for this case.
> Have you considered having some way to distribute the keys to all nodes such 
> that you don’t need to decrypt on the sending side?  Having to do this will 
> mean a lot more overhead for the sending side of a streaming operation.
>

Yes, I would likely fail the start when encryption is enabled and
there is no TLS between nodes and yes, zero copy streaming should not
be triggered here.

I have not considered that distribution. Honestly this seems like a
very complex setup. Due to the nature of Cassandra how one can easily
add / remove nodes, there would be a lot of hassle to distribute the
key of a new node to all other nodes somehow conveniently. I can't
even imagine how it would look in practice.

> > The second question is about key rotation. If an operator needs to
> > roll the key because it was compromised or there is some policy around
> > that, we should be able to provide some way to rotate it. Our idea is
> > to write a tool (either a subcommand of nodetool (rewritesstables)
> > command or a completely standalone one in tools) which would take the
> > first, original key, the second, new key and dir with sstables as
> > input and it would literally took the data and it would rewrite it to
> > the second set of sstables which would be encrypted with the second
> > key. What do you think about this?
>
> I would rather suggest that “what key encrypted this” be part of the sstable 
> metadata, and allow there to be multiple keys in the system.  This way you 
> can just add a new “current key” so new sstables use the new key, but 
> existing sstables would use the old key.  An operator could then trigger a 
> “nodetool upgradesstables —all” to rewrite the existing sstables with the new 
> “current key”.

How would this key be added when Cassanra runs? Via JMX? So that means
JMX itself has to be secure to send that key to it or it would not
make sense. Or adding a new key would mean a node needs to go down and
we would somehow configure it on startup? But if a node needs to go
down first, we can just rewrite these tables while it is offline and
there is no need to do it that way.

The tangential topic to this problem is if we are trying to do this
while the whole cluster is fully up and operational or we are ok to
bring that respective node down for a while to rewrite sstables for
it. I consider the "always up" scenario very complex to nail down
correctly and that is not a task I can do on my own with my current
understanding of Cassandra codebase.

> >
> > Regards
> >
> > On Sat, 13 Nov 2021 at 19:35,  wrote:
> >>
> >> Same reaction here - great to have traction on this ticket. Shylaja, 
> >> thanks for your work on this and to Stefan as well! It would be wonderful 
> >> to have the feature complete.
> >>
> >> One thing I’d mention is that a lot’s changed about the project’s testing 
> >> strategy since the original patch was written. I see that the 2016 version 
> >> adds a couple round-trip unit tests with a small amount of static data. It 
> >> would be good to see randomized tests fleshed out that exercise more of 
> >> the read/write path; or which add variants of existing read/write path 
> >> tests that enable encryption.
> >>
> >> – Scott
> >>
> >>> On Nov 13, 2021, at 7:53 AM, Brandon Williams  wrote:
> >>>
> >>> We already have a ticket and this predated CEPs, and being an
> >>> obviously good improvement to have that many have been asking for for
> >>> some time now, I don't see the need for a CEP here.
> >>>
> >>> On Sat, Nov 13, 2021 at 5:01 AM Stefan Miklosovic
> >>>  wrote:
> 
>  Hi list,
> 
>  an engineer from Intel - Shylaja Kokoori (who is watching this list
>  closely) has retrofitted the original code from CASSANDRA-9633 work in
>  times of 3.4 to the current trunk with my help here and there, mostly
>  cosmetic.
> 
>  I would like to know if there is a general consensus about me going to
>  create a CEP for this feature or what is your perception on this. I
>  know we have it a little bit backwards here as we should first discuss
>  and then code but I 

Re: [VOTE] CEP-17: SSTable format API

[bened...@apache.org]

+1

From: Brandon Williams 
Date: Monday, 15 November 2021 at 19:47
To: dev@cassandra.apache.org 
Subject: Re: [VOTE] CEP-17: SSTable format API
+1

On Mon, Nov 15, 2021 at 1:43 PM Branimir Lambov  wrote:
>
> Hi everyone,
>
> I would like to start a vote on this CEP.
>
> Proposal:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
>
> Discussion:
> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
>
> The vote will be open for 72 hours.
> A vote passes if there are at least three binding +1s and no binding vetoes.
>
> Regards,
> Branimir

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [VOTE] CEP-17: SSTable format API

[Jeff Jirsa]

+1


On Mon, Nov 15, 2021 at 11:43 AM Branimir Lambov  wrote:

> Hi everyone,
>
> I would like to start a vote on this CEP.
>
> Proposal:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
>
> Discussion:
>
> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
>
> The vote will be open for 72 hours.
> A vote passes if there are at least three binding +1s and no binding
> vetoes.
>
> Regards,
> Branimir
>

Re: [VOTE] CEP-17: SSTable format API

[Ekaterina Dimitrova]

+1

On Mon, 15 Nov 2021 at 15:51, bened...@apache.org 
wrote:

> +1
>
> From: Brandon Williams 
> Date: Monday, 15 November 2021 at 19:47
> To: dev@cassandra.apache.org 
> Subject: Re: [VOTE] CEP-17: SSTable format API
> +1
>
> On Mon, Nov 15, 2021 at 1:43 PM Branimir Lambov 
> wrote:
> >
> > Hi everyone,
> >
> > I would like to start a vote on this CEP.
> >
> > Proposal:
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
> >
> > Discussion:
> >
> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
> >
> > The vote will be open for 72 hours.
> > A vote passes if there are at least three binding +1s and no binding
> vetoes.
> >
> > Regards,
> > Branimir
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

Re: [VOTE] CEP-17: SSTable format API

[David Capwell]

+1

> On Nov 15, 2021, at 12:18 PM, bened...@apache.org wrote:
> 
> +1
> 
> From: Brandon Williams 
> Date: Monday, 15 November 2021 at 19:47
> To: dev@cassandra.apache.org 
> Subject: Re: [VOTE] CEP-17: SSTable format API
> +1
> 
> On Mon, Nov 15, 2021 at 1:43 PM Branimir Lambov  wrote:
>> 
>> Hi everyone,
>> 
>> I would like to start a vote on this CEP.
>> 
>> Proposal:
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
>> 
>> Discussion:
>> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
>> 
>> The vote will be open for 72 hours.
>> A vote passes if there are at least three binding +1s and no binding vetoes.
>> 
>> Regards,
>> Branimir
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [VOTE] CEP-17: SSTable format API

[J. D. Jordan]

+1 nb

> On Nov 15, 2021, at 1:47 PM, Brandon Williams  wrote:
> 
> +1
> 
>> On Mon, Nov 15, 2021 at 1:43 PM Branimir Lambov  wrote:
>> 
>> Hi everyone,
>> 
>> I would like to start a vote on this CEP.
>> 
>> Proposal:
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
>> 
>> Discussion:
>> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
>> 
>> The vote will be open for 72 hours.
>> A vote passes if there are at least three binding +1s and no binding vetoes.
>> 
>> Regards,
>> Branimir
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Resurrection of CASSANDRA-9633 - SSTable encryption

[Jeremiah D Jordan]

> On Nov 15, 2021, at 2:25 PM, Stefan Miklosovic 
>  wrote:
> 
> On Mon, 15 Nov 2021 at 19:42, Jeremiah D Jordan
> mailto:jeremiah.jor...@gmail.com>> wrote:
>> 
>> 
>> 
>>> On Nov 14, 2021, at 3:53 PM, Stefan Miklosovic 
>>>  wrote:
>>> 
>>> Hey,
>>> 
>>> there are two points we are not completely sure about.
>>> 
>>> The first one is streaming. If there is a cluster of 5 nodes, each
>>> node has its own unique encryption key. Hence, if a SSTable is stored
>>> on a disk with the key for node 1 and this is streamed to node 2 -
>>> which has a different key - it would not be able to decrypt that. Our
>>> idea is to actually send data over the wire _decrypted_ however it
>>> would be still secure if internode communication is done via TLS. Is
>>> this approach good with you?
>>> 
>> 
>> So would you fail startup if someone enabled sstable encryption but did not 
>> have TLS for internode communication?  Another concern here is making sure 
>> zero copy streaming does not get triggered for this case.
>> Have you considered having some way to distribute the keys to all nodes such 
>> that you don’t need to decrypt on the sending side?  Having to do this will 
>> mean a lot more overhead for the sending side of a streaming operation.
>> 
> 
> Yes, I would likely fail the start when encryption is enabled and
> there is no TLS between nodes and yes, zero copy streaming should not
> be triggered here.
> 
> I have not considered that distribution. Honestly this seems like a
> very complex setup. Due to the nature of Cassandra how one can easily
> add / remove nodes, there would be a lot of hassle to distribute the
> key of a new node to all other nodes somehow conveniently. I can't
> even imagine how it would look in practice.

One since the default implementation seems to be to just read the key out of a 
keystore, you could require having the same keystore on every node, and then it 
could just be the same key used by all nodes.  Or there could actually be a 
different key for each node as the “key_alias” in the yaml, but every node has 
all the other nodes keys in its local keystore still.  When adding a new node 
with a new key the operator would have to go add the new key to each existing 
keystore before adding the new node to the cluster.

Another method could be to get the keys from a key server rather than a local 
file.

But yes decrypt before streaming, stream encrypted using TLS, re-encrypt before 
writing to disk is an option.  Just one that requires losing all the 
performance advantages gained when “do not decompress streaming” and then later 
“zero copy streaming” were implemented.

> 
>>> The second question is about key rotation. If an operator needs to
>>> roll the key because it was compromised or there is some policy around
>>> that, we should be able to provide some way to rotate it. Our idea is
>>> to write a tool (either a subcommand of nodetool (rewritesstables)
>>> command or a completely standalone one in tools) which would take the
>>> first, original key, the second, new key and dir with sstables as
>>> input and it would literally took the data and it would rewrite it to
>>> the second set of sstables which would be encrypted with the second
>>> key. What do you think about this?
>> 
>> I would rather suggest that “what key encrypted this” be part of the sstable 
>> metadata, and allow there to be multiple keys in the system.  This way you 
>> can just add a new “current key” so new sstables use the new key, but 
>> existing sstables would use the old key.  An operator could then trigger a 
>> “nodetool upgradesstables —all” to rewrite the existing sstables with the 
>> new “current key”.
> 
> How would this key be added when Cassanra runs? Via JMX? So that means
> JMX itself has to be secure to send that key to it or it would not
> make sense. Or adding a new key would mean a node needs to go down and
> we would somehow configure it on startup? But if a node needs to go
> down first, we can just rewrite these tables while it is offline and
> there is no need to do it that way.

Given the current comments in the cassandra.yaml I would expect it to work like 
I just said:
https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L1328-L1332 


# Enables encrypting data at-rest (on disk). Different key providers can be 
plugged in, but the default reads from
# a JCE-style keystore. A single keystore can hold multiple keys, but the one 
referenced by
# the "key_alias" is the only key that will be used for encrypt opertaions; 
previously used keys
# can still (and should!) be in the keystore and will be used on decrypt 
operations
# (to handle the case of key rotation).

The intended method to rotate keys from that comments seems to be:
1. Stop the node
2. Adds a new key to the keystore
3. Changes the “key_alias” to reference the new key
4. Start the node

All newly encrypted things would use the new 

Re: Resurrection of CASSANDRA-9633 - SSTable encryption

[bened...@apache.org]

If decrypting before transmission we’ll want to require the cluster to have an 
internode authenticator setup, else a nefarious process could simply ask for 
data to be streamed to it to circumvent the encryption.

I agree it would be nice to have the nodes share the secret some way to avoid 
the additional performance penalties and potential security issues associated 
with decryption/encryption for transmission, but I don’t have any practical 
experience securely sharing secrets between nodes, so I’ll leave that decision 
to those who do.


From: Jeremiah D Jordan 
Date: Monday, 15 November 2021 at 22:09
To: dev@cassandra.apache.org 
Subject: Re: Resurrection of CASSANDRA-9633 - SSTable encryption


> On Nov 15, 2021, at 2:25 PM, Stefan Miklosovic 
>  wrote:
>
> On Mon, 15 Nov 2021 at 19:42, Jeremiah D Jordan
> mailto:jeremiah.jor...@gmail.com>> wrote:
>>
>>
>>
>>> On Nov 14, 2021, at 3:53 PM, Stefan Miklosovic 
>>>  wrote:
>>>
>>> Hey,
>>>
>>> there are two points we are not completely sure about.
>>>
>>> The first one is streaming. If there is a cluster of 5 nodes, each
>>> node has its own unique encryption key. Hence, if a SSTable is stored
>>> on a disk with the key for node 1 and this is streamed to node 2 -
>>> which has a different key - it would not be able to decrypt that. Our
>>> idea is to actually send data over the wire _decrypted_ however it
>>> would be still secure if internode communication is done via TLS. Is
>>> this approach good with you?
>>>
>>
>> So would you fail startup if someone enabled sstable encryption but did not 
>> have TLS for internode communication?  Another concern here is making sure 
>> zero copy streaming does not get triggered for this case.
>> Have you considered having some way to distribute the keys to all nodes such 
>> that you don’t need to decrypt on the sending side?  Having to do this will 
>> mean a lot more overhead for the sending side of a streaming operation.
>>
>
> Yes, I would likely fail the start when encryption is enabled and
> there is no TLS between nodes and yes, zero copy streaming should not
> be triggered here.
>
> I have not considered that distribution. Honestly this seems like a
> very complex setup. Due to the nature of Cassandra how one can easily
> add / remove nodes, there would be a lot of hassle to distribute the
> key of a new node to all other nodes somehow conveniently. I can't
> even imagine how it would look in practice.

One since the default implementation seems to be to just read the key out of a 
keystore, you could require having the same keystore on every node, and then it 
could just be the same key used by all nodes.  Or there could actually be a 
different key for each node as the “key_alias” in the yaml, but every node has 
all the other nodes keys in its local keystore still.  When adding a new node 
with a new key the operator would have to go add the new key to each existing 
keystore before adding the new node to the cluster.

Another method could be to get the keys from a key server rather than a local 
file.

But yes decrypt before streaming, stream encrypted using TLS, re-encrypt before 
writing to disk is an option.  Just one that requires losing all the 
performance advantages gained when “do not decompress streaming” and then later 
“zero copy streaming” were implemented.

>
>>> The second question is about key rotation. If an operator needs to
>>> roll the key because it was compromised or there is some policy around
>>> that, we should be able to provide some way to rotate it. Our idea is
>>> to write a tool (either a subcommand of nodetool (rewritesstables)
>>> command or a completely standalone one in tools) which would take the
>>> first, original key, the second, new key and dir with sstables as
>>> input and it would literally took the data and it would rewrite it to
>>> the second set of sstables which would be encrypted with the second
>>> key. What do you think about this?
>>
>> I would rather suggest that “what key encrypted this” be part of the sstable 
>> metadata, and allow there to be multiple keys in the system.  This way you 
>> can just add a new “current key” so new sstables use the new key, but 
>> existing sstables would use the old key.  An operator could then trigger a 
>> “nodetool upgradesstables —all” to rewrite the existing sstables with the 
>> new “current key”.
>
> How would this key be added when Cassanra runs? Via JMX? So that means
> JMX itself has to be secure to send that key to it or it would not
> make sense. Or adding a new key would mean a node needs to go down and
> we would somehow configure it on startup? But if a node needs to go
> down first, we can just rewrite these tables while it is offline and
> there is no need to do it that way.

Given the current comments in the cassandra.yaml I would expect it to work like 
I just said:
https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L1328-L1332 

Re: Resurrection of CASSANDRA-9633 - SSTable encryption

[Bowen Song]

The second question is about key rotation. If an operator needs to
roll the key because it was compromised or there is some policy around
that, we should be able to provide some way to rotate it. Our idea is
to write a tool (either a subcommand of nodetool (rewritesstables)
command or a completely standalone one in tools) which would take the
first, original key, the second, new key and dir with sstables as
input and it would literally took the data and it would rewrite it to
the second set of sstables which would be encrypted with the second
key. What do you think about this?


   I would rather suggest that “what key encrypted this” be part of the sstable 
metadata, and allow there to be multiple keys in the system.  This way you can 
just add a new “current key” so new sstables use the new key, but existing 
sstables would use the old key.  An operator could then trigger a “nodetool 
upgradesstables —all” to rewrite the existing sstables with the new “current 
key”.

There's a much better approach to solve this issue. You can stored a 
wrapped key in an encryption info file alone side the SSTable file. 
Here's how it works:

1. randomly generate a key Kr
2. encrypt the SSTable file with the key Kr, store the encrypted SSTable 
file on disk
3. derive a key encryption key KEK from the SSTable file's information 
(e.g.: table UUID + generation) and the user chosen master key Km, so 
you have KEK = KDF(UUID+GEN, Km)

4. wrap (encrypt) the key Kr with the KEK, so you have WKr = KW(Kr, KEK)
5. hash the Km, the hash will used as a key ID to identify which master 
key was used to encrypt the key Kr if the server has multiple master 
keys in use
6. store the the WKr and the hash of Km in a separate file alone side 
the SSTable file


In the read path, the Kr should be kept in memory to help improve 
performance and this will also allow zero-downtime master key rotation.


During a key rotation:
1. derive the KEK in the same way: KEK = KDF(UUID+GEN, Km)
2. read the WKr from the encryption information file, and unwrap 
(decrypt) it using the KEK to get the Kr

3. derive a new KEK' from the new master key Km' in the same way as above
4. wrap (encrypt) the key Kr with KEK' to get WKr' = KW(Kr, KEK')
5. hash the new master key Km', and store it together with the WKr' in 
the encryption info file


Since the key rotation only involves rewriting the encryption info file, 
the operation should take only a few milliseconds per SSTable file, it 
will be much faster than decrypting and then re-encrypting the SSTable data.




On 15/11/2021 18:42, Jeremiah D Jordan wrote:



On Nov 14, 2021, at 3:53 PM, Stefan 
Miklosovic  wrote:

Hey,

there are two points we are not completely sure about.

The first one is streaming. If there is a cluster of 5 nodes, each
node has its own unique encryption key. Hence, if a SSTable is stored
on a disk with the key for node 1 and this is streamed to node 2 -
which has a different key - it would not be able to decrypt that. Our
idea is to actually send data over the wire _decrypted_ however it
would be still secure if internode communication is done via TLS. Is
this approach good with you?


So would you fail startup if someone enabled sstable encryption but did not 
have TLS for internode communication?  Another concern here is making sure zero 
copy streaming does not get triggered for this case.
Have you considered having some way to distribute the keys to all nodes such 
that you don’t need to decrypt on the sending side?  Having to do this will 
mean a lot more overhead for the sending side of a streaming operation.


The second question is about key rotation. If an operator needs to
roll the key because it was compromised or there is some policy around
that, we should be able to provide some way to rotate it. Our idea is
to write a tool (either a subcommand of nodetool (rewritesstables)
command or a completely standalone one in tools) which would take the
first, original key, the second, new key and dir with sstables as
input and it would literally took the data and it would rewrite it to
the second set of sstables which would be encrypted with the second
key. What do you think about this?

I would rather suggest that “what key encrypted this” be part of the sstable 
metadata, and allow there to be multiple keys in the system.  This way you can 
just add a new “current key” so new sstables use the new key, but existing 
sstables would use the old key.  An operator could then trigger a “nodetool 
upgradesstables —all” to rewrite the existing sstables with the new “current 
key”.


Regards

On Sat, 13 Nov 2021 at 19:35,  wrote:

Same reaction here - great to have traction on this ticket. Shylaja, thanks for 
your work on this and to Stefan as well! It would be wonderful to have the 
feature complete.

One thing I’d mention is that a lot’s changed about the project’s testing 
strategy since the original patch was written. I see that the 2016 version ad

Re: Resurrection of CASSANDRA-9633 - SSTable encryption

[J. D. Jordan]

Another comment here. I tried to find the patch to check but couldn’t find it 
linked to the ticket. If it is not already, given the TDE key class is 
pluggable in the yaml, when a file is written everything need to instantiate 
the class to decrypt it should be in the metadata. Just like happens now for 
compression. So if someone switches to a different TDE class you can still know 
to instantiate the old one to read existing files.  The class from the yaml 
config should just be used for encrypting new files.

> On Nov 14, 2021, at 3:54 PM, Stefan Miklosovic 
>  wrote:
> 
> Hey,
> 
> there are two points we are not completely sure about.
> 
> The first one is streaming. If there is a cluster of 5 nodes, each
> node has its own unique encryption key. Hence, if a SSTable is stored
> on a disk with the key for node 1 and this is streamed to node 2 -
> which has a different key - it would not be able to decrypt that. Our
> idea is to actually send data over the wire _decrypted_ however it
> would be still secure if internode communication is done via TLS. Is
> this approach good with you?
> 
> The second question is about key rotation. If an operator needs to
> roll the key because it was compromised or there is some policy around
> that, we should be able to provide some way to rotate it. Our idea is
> to write a tool (either a subcommand of nodetool (rewritesstables)
> command or a completely standalone one in tools) which would take the
> first, original key, the second, new key and dir with sstables as
> input and it would literally took the data and it would rewrite it to
> the second set of sstables which would be encrypted with the second
> key. What do you think about this?
> 
> Regards
> 
>> On Sat, 13 Nov 2021 at 19:35,  wrote:
>> 
>> Same reaction here - great to have traction on this ticket. Shylaja, thanks 
>> for your work on this and to Stefan as well! It would be wonderful to have 
>> the feature complete.
>> 
>> One thing I’d mention is that a lot’s changed about the project’s testing 
>> strategy since the original patch was written. I see that the 2016 version 
>> adds a couple round-trip unit tests with a small amount of static data. It 
>> would be good to see randomized tests fleshed out that exercise more of the 
>> read/write path; or which add variants of existing read/write path tests 
>> that enable encryption.
>> 
>> – Scott
>> 
 On Nov 13, 2021, at 7:53 AM, Brandon Williams  wrote:
>>> 
>>> We already have a ticket and this predated CEPs, and being an
>>> obviously good improvement to have that many have been asking for for
>>> some time now, I don't see the need for a CEP here.
>>> 
>>> On Sat, Nov 13, 2021 at 5:01 AM Stefan Miklosovic
>>>  wrote:
 
 Hi list,
 
 an engineer from Intel - Shylaja Kokoori (who is watching this list
 closely) has retrofitted the original code from CASSANDRA-9633 work in
 times of 3.4 to the current trunk with my help here and there, mostly
 cosmetic.
 
 I would like to know if there is a general consensus about me going to
 create a CEP for this feature or what is your perception on this. I
 know we have it a little bit backwards here as we should first discuss
 and then code but I am super glad that we have some POC we can
 elaborate further on and CEP would just cement  and summarise the
 approach / other implementation aspects of this feature.
 
 I think that having 9633 merged will fill quite a big operational gap
 when it comes to security. There are a lot of enterprises who desire
 this feature so much. I can not remember when I last saw a ticket with
 50 watchers which was inactive for such a long time.
 
 Regards
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
 For additional commands, e-mail: dev-h...@cassandra.apache.org
 
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [VOTE] CEP-17: SSTable format API

[Nate McCall]

+1


On Tue, Nov 16, 2021 at 8:43 AM Branimir Lambov  wrote:

> Hi everyone,
>
> I would like to start a vote on this CEP.
>
> Proposal:
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API
>
> Discussion:
>
> https://lists.apache.org/thread.html/r636bebcab4e678dbee042285449193e8e75d3753200a1b404fcc7196%40%3Cdev.cassandra.apache.org%3E
>
> The vote will be open for 72 hours.
> A vote passes if there are at least three binding +1s and no binding
> vetoes.
>
> Regards,
> Branimir
>