Re: [DISCUSS] Limiting query results by size (CASSANDRA-11745)

[Benjamin Lerer]

>
> So my other question - for aggregation with the "group by" clause, we
> return an aggregated row which is computed from a group of rows - with my
> current implementation, it is approximated by counting the size of the
> largest row in that group - I think it is the safest and simplest
> approximation - wdyt?


I feel that there are something that was not discussed here. The storage
engine can return some rows that are much larger than the actual row
returned to the user depending on the projections being used. Therefore
there will only be a reliable matching between the size of the page loaded
internally and the size of the page returned to the user when the full row
is queried without transformation. For all the other case the difference
can be really significant. For a group by queries doing a count(*), the
approach suggested will return a page size that is totally off with what
was requested.

Le mar. 13 juin 2023 à 07:00, Jacek Lewandowski 
a écrit :

> Josh, that answers my question exactly; thank you.
>
> I will not implement limiting the result set in CQL (that is, by LIMIT
> clause) and stay with just paging. Whether the page size is defined in
> bytes or rows can be determined by a flag - there are many unused bits for
> that.
>
> So my other question - for aggregation with the "group by" clause, we
> return an aggregated row which is computed from a group of rows - with my
> current implementation, it is approximated by counting the size of the
> largest row in that group - I think it is the safest and simplest
> approximation - wdyt?
>
>
> pon., 12 cze 2023 o 22:55 Josh McKenzie  napisał(a):
>
>> As long as it is valid in the paging protocol to return a short page, but
>> still say “there are more pages”, I think that is fine to do that.
>>
>> Thankfully the v3-v5 spec all make it clear that clients need to respect
>> what the server has to say about there being more pages:
>> https://github.com/apache/cassandra/blob/trunk/doc/native_protocol_v5.spec#L1247-L1253
>>
>>   - Clients should not rely on the actual size of the result set returned
>> to
>> decide if there are more results to fetch or not. Instead, they
>> should always
>> check the Has_more_pages flag (unless they did not enable paging for
>> the query
>> obviously). Clients should also not assert that no result will have
>> more than
>>  results. While the current implementation always
>> respects
>> the exact value of , we reserve the right to return
>> slightly smaller or bigger pages in the future for performance
>> reasons.
>>
>>
>> On Mon, Jun 12, 2023, at 3:19 PM, Jeremiah Jordan wrote:
>>
>> As long as it is valid in the paging protocol to return a short page, but
>> still say “there are more pages”, I think that is fine to do that.  For an
>> actual LIMIT that is part of the user query, I think the server must always
>> have returned all data that fits into the LIMIT when all pages have been
>> returned.
>>
>> -Jeremiah
>>
>> On Jun 12, 2023 at 12:56:14 PM, Josh McKenzie 
>> wrote:
>>
>>
>> Yeah, my bad. I have paging on the brain. Seriously.
>>
>> I can't think of a use-case in which a LIMIT based on # bytes makes sense
>> from a user perspective.
>>
>> On Mon, Jun 12, 2023, at 1:35 PM, Jeff Jirsa wrote:
>>
>>
>>
>> On Mon, Jun 12, 2023 at 9:50 AM Benjamin Lerer  wrote:
>>
>> If you have rows that vary significantly in their size, your latencies
>> could end up being pretty unpredictable using a LIMIT BY . Being
>> able to specify a limit by bytes at the driver / API level would allow app
>> devs to get more deterministic results out of their interaction w/the DB if
>> they're looking to respond back to a client within a certain time frame and
>> / or determine next steps in the app (continue paging, stop, etc) based on
>> how long it took to get results back.
>>
>>
>> Are you talking about the page size or the LIMIT. Once the LIMIT is
>> reached there is no "continue paging". LIMIT is also at the CQL level not
>> at the driver level.
>> I can totally understand the need for a page size in bytes not for a
>> LIMIT.
>>
>>
>> Would only ever EXPECT to see a page size in bytes, never a LIMIT
>> specifying bytes.
>>
>> I know the C-11745 ticket says LIMIT, too, but that feels very odd to me.
>>
>>
>>
>>

[VOTE] CEP-8 Datastax Drivers Donation

[Jeremy Hanna]

Calling for a vote on CEP-8 [1].

To clarify the intent, as Benjamin said in the discussion thread [2], the goal 
of this vote is simply to ensure that the community is in favor of the 
donation. Nothing more.
The plan is to introduce the drivers, one by one. Each driver donation will 
need to be accepted first by the PMC members, as it is the case for any 
donation. Therefore the PMC should have full control on the pace at which new 
drivers are accepted.

If this vote passes, we can start this process for the Java driver under the 
direction of the PMC.

Jeremy

1. 
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Brandon Williams]

+1

Kind Regards,
Brandon

On Tue, Jun 13, 2023 at 9:15 AM Jeremy Hanna  wrote:
>
> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the 
> goal of this vote is simply to ensure that the community is in favor of the 
> donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation will 
> need to be accepted first by the PMC members, as it is the case for any 
> donation. Therefore the PMC should have full control on the pace at which new 
> drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under the 
> direction of the PMC.
>
> Jeremy
>
> 1. 
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Paulo Motta]

+1

On Tue, Jun 13, 2023 at 10:16 AM Brandon Williams  wrote:

> +1
>
> Kind Regards,
> Brandon
>
> On Tue, Jun 13, 2023 at 9:15 AM Jeremy Hanna 
> wrote:
> >
> > Calling for a vote on CEP-8 [1].
> >
> > To clarify the intent, as Benjamin said in the discussion thread [2],
> the goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> > The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
> >
> > If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
> >
> > Jeremy
> >
> > 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> > 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Jeff Jirsa]

+1


On Tue, Jun 13, 2023 at 7:15 AM Jeremy Hanna 
wrote:

> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the
> goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
>
> Jeremy
>
> 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Ekaterina Dimitrova]

+1

On Tue, 13 Jun 2023 at 10:33, Jeff Jirsa  wrote:

> +1
>
>
> On Tue, Jun 13, 2023 at 7:15 AM Jeremy Hanna 
> wrote:
>
>> Calling for a vote on CEP-8 [1].
>>
>> To clarify the intent, as Benjamin said in the discussion thread [2], the
>> goal of this vote is simply to ensure that the community is in favor of
>> the donation. Nothing more.
>> The plan is to introduce the drivers, one by one. Each driver donation
>> will need to be accepted first by the PMC members, as it is the case for
>> any donation. Therefore the PMC should have full control on the pace at
>> which new drivers are accepted.
>>
>> If this vote passes, we can start this process for the Java driver under
>> the direction of the PMC.
>>
>> Jeremy
>>
>> 1.
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>>
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Benjamin Lerer]

+1

Le mar. 13 juin 2023 à 16:46, Ekaterina Dimitrova  a
écrit :

> +1
>
> On Tue, 13 Jun 2023 at 10:33, Jeff Jirsa  wrote:
>
>> +1
>>
>>
>> On Tue, Jun 13, 2023 at 7:15 AM Jeremy Hanna 
>> wrote:
>>
>>> Calling for a vote on CEP-8 [1].
>>>
>>> To clarify the intent, as Benjamin said in the discussion thread [2],
>>> the goal of this vote is simply to ensure that the community is in
>>> favor of the donation. Nothing more.
>>> The plan is to introduce the drivers, one by one. Each driver donation
>>> will need to be accepted first by the PMC members, as it is the case for
>>> any donation. Therefore the PMC should have full control on the pace at
>>> which new drivers are accepted.
>>>
>>> If this vote passes, we can start this process for the Java driver under
>>> the direction of the PMC.
>>>
>>> Jeremy
>>>
>>> 1.
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>>>
>>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Mick Semb Wever]

Calling for a vote on CEP-8 [1].
>


+1 to the CEP.

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Jeremiah Jordan]

 +1 nb

On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
wrote:

> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the
> goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
>
> Jeremy
>
> 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>

Re: [DISCUSS] Remove deprecated keyspace_count_warn_threshold and table_count_warn_threshold

[Josh McKenzie]

> have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when 
> they were replaced/migrated to guardrails as part of CEP-3 (Guardrails).
Have we been dropping support entirely for old params or using the @Replaces 
annotation into perpetuity?

I dislike the idea of operators having to remember to update things between 
versions and being surprised when things change roughly equally to us carrying 
along undocumented deprecated param name mapping roughly equally. :)

On Mon, Jun 12, 2023, at 5:56 PM, Dan Jatnieks wrote:
> Hello everyone,
> 
> I would like to propose removing the non-guardrail thresholds 
> 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' 
> configuration settings on the trunk branch for the next major release.
> 
> These thresholds were first added with CASSANDRA-16309 in 4.0-beta4 and have 
> subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when they 
> were replaced/migrated to guardrails as part of CEP-3 (Guardrails).
> 
> I'd appreciate any thoughts about this. I will open a ticket to get started 
> if there is support for doing this.
> 
> Reference:
> https://issues.apache.org/jira/browse/CASSANDRA-16309
> https://issues.apache.org/jira/browse/CASSANDRA-17195
> CEP-3: Guardrails 
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails
> 
> 
> Thanks,
> Dan Jatnieks
> 

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Josh McKenzie]

+1

On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
> +1 nb
> 
> On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna  
> wrote:
>> 
>> Calling for a vote on CEP-8 [1].
>> 
>> To clarify the intent, as Benjamin said in the discussion thread [2], the 
>> goal of this vote is simply to ensure that the community is in favor of the 
>> donation. Nothing more.
>> The plan is to introduce the drivers, one by one. Each driver donation will 
>> need to be accepted first by the PMC members, as it is the case for any 
>> donation. Therefore the PMC should have full control on the pace at which 
>> new drivers are accepted.
>> 
>> If this vote passes, we can start this process for the Java driver under the 
>> direction of the PMC.
>> 
>> Jeremy
>> 
>> 1. 
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [VOTE] CEP-8 Datastax Drivers Donation

[David Capwell]

+1

> On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:
> 
> +1
> 
> On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
>> +1 nb
>> 
>> On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna > > wrote:
>>> 
>>> Calling for a vote on CEP-8 [1].
>>> 
>>> To clarify the intent, as Benjamin said in the discussion thread [2], the 
>>> goal of this vote is simply to ensure that the community is in favor of the 
>>> donation. Nothing more.
>>> The plan is to introduce the drivers, one by one. Each driver donation will 
>>> need to be accepted first by the PMC members, as it is the case for any 
>>> donation. Therefore the PMC should have full control on the pace at which 
>>> new drivers are accepted.
>>> 
>>> If this vote passes, we can start this process for the Java driver under 
>>> the direction of the PMC.
>>> 
>>> Jeremy
>>> 
>>> 1. 
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Yifan Cai]

+1

From: David Capwell 
Sent: Tuesday, June 13, 2023 8:37:10 AM
To: dev 
Subject: Re: [VOTE] CEP-8 Datastax Drivers Donation

+1

On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:

+1

On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
+1 nb

On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
mailto:jeremy.hanna1...@gmail.com>> wrote:

Calling for a vote on CEP-8 [1].

To clarify the intent, as Benjamin said in the discussion thread [2], the goal 
of this vote is simply to ensure that the community is in favor of the 
donation. Nothing more.
The plan is to introduce the drivers, one by one. Each driver donation will 
need to be accepted first by the PMC members, as it is the case for any 
donation. Therefore the PMC should have full control on the pace at which new 
drivers are accepted.

If this vote passes, we can start this process for the Java driver under the 
direction of the PMC.

Jeremy

1. 
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [DISCUSS] Remove deprecated keyspace_count_warn_threshold and table_count_warn_threshold

[David Capwell]

> Have we been dropping support entirely for old params or using the @Replaces 
> annotation into perpetuity?


My understanding is that the goal is to keep things around in perpetuity unless 
it actively causes us harm… and with @Replaces, there tends to be no harm to 
keep around…

Looking at 
https://github.com/apache/cassandra/commit/bae92ee139b411c94228f8fd5bb8befb4183ca9f
 we just marked them deprecated and created a brand new config that matched the 
old… which I feel was a bad idea…. Renaming configs are fine with @Replaces, 
but asking users to migrate with the idea of breaking them in the future is bad…

The table_count_warn_threshold config is used at 
org.apache.cassandra.cql3.statements.schema.CreateTableStatement#clientWarnings
The tables_warn_threshold config is used at 
org.apache.cassandra.cql3.statements.schema.CreateTableStatement#validate

The only difference I see is that table_count_warn_threshold includes system 
tables where as tables_warn_threshold is only user tables…

> I would like to propose removing the non-guardrail thresholds 
> 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' 
> configuration settings on the trunk branch for the next major release.

Deprecate in 4.1 is way too new for me to accept that, and its low effort to 
keep; breaking users is always a bad idea and doing it when not needed is bad…

Honestly, I don’t see why we couldn’t use @Replaces here to solve the semantic 
gap… table_count_warn_threshold includes the system tables, so we just need a 
Converter that takes w/e the value the user put in and subtracts the system 
tables… which then gives us the user tables (matching tables_warn_threshold)

> On Jun 13, 2023, at 7:57 AM, Josh McKenzie  wrote:
> 
>> have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when 
>> they were replaced/migrated to guardrails as part of CEP-3 (Guardrails).
> Have we been dropping support entirely for old params or using the @Replaces 
> annotation into perpetuity?
> 
> I dislike the idea of operators having to remember to update things between 
> versions and being surprised when things change roughly equally to us 
> carrying along undocumented deprecated param name mapping roughly equally. :)
> 
> On Mon, Jun 12, 2023, at 5:56 PM, Dan Jatnieks wrote:
>> Hello everyone,
>> 
>> I would like to propose removing the non-guardrail thresholds 
>> 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' 
>> configuration settings on the trunk branch for the next major release.
>> 
>> These thresholds were first added with CASSANDRA-16309 in 4.0-beta4 and have 
>> subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when they 
>> were replaced/migrated to guardrails as part of CEP-3 (Guardrails).
>> 
>> I'd appreciate any thoughts about this. I will open a ticket to get started 
>> if there is support for doing this.
>> 
>> Reference:
>> https://issues.apache.org/jira/browse/CASSANDRA-16309
>> https://issues.apache.org/jira/browse/CASSANDRA-17195
>> CEP-3: Guardrails 
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails
>> 
>> 
>> Thanks,
>> Dan Jatnieks

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Andrés de la Peña]

+1

On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:

> +1
> --
> *From:* David Capwell 
> *Sent:* Tuesday, June 13, 2023 8:37:10 AM
> *To:* dev 
> *Subject:* Re: [VOTE] CEP-8 Datastax Drivers Donation
>
> +1
>
> On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:
>
> +1
>
> On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
>
> +1 nb
>
> On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
> wrote:
>
>
> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the
> goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
>
> Jeremy
>
> 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>
>
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Francisco Guerrero]

+1 (nb)

On 2023/06/13 16:22:55 Andrés de la Peña wrote:
> +1
> 
> On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:
> 
> > +1
> > --
> > *From:* David Capwell 
> > *Sent:* Tuesday, June 13, 2023 8:37:10 AM
> > *To:* dev 
> > *Subject:* Re: [VOTE] CEP-8 Datastax Drivers Donation
> >
> > +1
> >
> > On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:
> >
> > +1
> >
> > On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
> >
> > +1 nb
> >
> > On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
> > wrote:
> >
> >
> > Calling for a vote on CEP-8 [1].
> >
> > To clarify the intent, as Benjamin said in the discussion thread [2], the
> > goal of this vote is simply to ensure that the community is in favor of
> > the donation. Nothing more.
> > The plan is to introduce the drivers, one by one. Each driver donation
> > will need to be accepted first by the PMC members, as it is the case for
> > any donation. Therefore the PMC should have full control on the pace at
> > which new drivers are accepted.
> >
> > If this vote passes, we can start this process for the Java driver under
> > the direction of the PMC.
> >
> > Jeremy
> >
> > 1.
> > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> > 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
> >
> >
> >
> 

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

[Maxim Muzafarov]

Hello everyone,

I have created the following JIRA issues to follow up on the
discussion and to improve the user experience with virtual tables.
I'll try to address them before the next release.

CASSANDRA-18586: CQLSH formatting output is incorrect when querying
the system properties virtual table
https://issues.apache.org/jira/browse/CASSANDRA-18586

CASSANDRA-18587: Align all system property names used by the project
to use the 'cassandra' prefix
https://issues.apache.org/jira/browse/CASSANDRA-18587


I want also to clarify and discuss with you some points related to the
SystemPropertiesTable virtual table.

1. Hide non-production environment properties in the SystemProperties
virtual table.

As you may know, the result of the query on the virtual table
currently includes the environment properties related to tests as well
(used internally for our testing purposes) ~ 42 out of 290. This seems
a bit redundant for production use and floods the query output with
the things you don't need to think about. I think we can add a new
property -Dcassandra.vt.show.test.system.properties (false by default)
properties to hide these test-related properties for production
environments while still using them for our test runs and build
scripts. Hiding test properties is not a regression in this case. Any
thoughts?


2. The absence of a description of the system properties.

I have found that there is no good description of the system
properties used to configure production environments. Is there any
documentation for this case? Do we need to keep these pages up to
date?

Here are some links I found:
https://docs.datastax.com/en/dse/6.8/dse-dev/datastax_enterprise/config/cassandraSystemProperties.html
https://cassandra.apache.org/doc/4.1/cassandra/getting_started/configuring.html#environment-variables

On Thu, 18 May 2023 at 16:03, Miklosovic, Stefan
 wrote:
>
> Hi Maxim,
>
> thanks for bringing this up. I am glad you did the heavy-lifting in / around 
> CassandraRelevantProperties and we can build on top of this.
>
> I am fine with @Replaces for Cassandra system properties. After we put 
> everything into CassandraRelevantProperties, one can easily see that there 
> are great inconsistencies in properties' naming. As we still need to support 
> the old names too, using @Replaces, the similar mechanism we used in 
> DatabaseDescriptor, seems like the ideal solution.
>
> By the way, when somebody queries system_views.system_properties, it looks 
> very strange in CQL shell, the formatting is just broken. EXPAND ON; does not 
> help either. It is quite hard to parse this visually if a user wants to see 
> them all. The reason is that there is "java.class.path" property for which 
> the value is so long that it basically breaks the output.
>
> Another solution would be to fix the output but I am not sure how it would 
> look like.
>
> As we are going to rename them to have same prefixes, could not we remodel 
> that table as well? For example:
>
> https://gist.github.com/smiklosovic/de662b7faa25e1fdd56805cdb5ba80a7
>
> Feel free to come up with a different approach.
>
> By doing this, it would be way easier to get just Cassandra properties or 
> just properties for tests or just Java properties and selecting just the 
> first two groups would not break CQLSH. It is nice that it would have same 
> prefix but I am trying to find a way how to utilize the same prefix in CQLSH 
> as well.
>
> 
> From: Maxim Muzafarov 
> Sent: Thursday, May 18, 2023 12:54
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSS] Moving system property names to the 
> CassandraRelevantProperties
>
> NetApp Security WARNING: This is an external email. Do not click links or 
> open attachments unless you recognize the sender and know the content is safe.
>
>
>
>
> Hello everyone,
>
>
> Thanks for following this thread and the review, all the system
> properties have been moved to CassandraRelevantProperties.
> So you can find out what it looks like from the following link:
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/CassandraRelevantProperties.java#L38
>
>
> I would like to show you a few more steps in this thread so that the
> solution is generally complete. As you may have noticed, we have three
> types of system properties: cassandra properties used in production
> environments, cassandra properties used for testing only, and
> non-cassandra properties. I would like to reuse the @Replaces
> annotation to rename cassandra-related properties according to the
> following pattern: the 'cassandra.' prefix is for production
> properties, and the 'cassandra.test' prefix is for testing properties.
>
> This makes the results of the SystemPropertiesTable virtual table look
> more natural to users. I thinks we should include this change for the
> 5.0 release.
> WDYT?
>
>
> The other code clarity minor improvements to do:
>
> 1.
> Use WithProperties to ensure that system propert

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Caleb Rackliffe]

+1

On Tue, Jun 13, 2023 at 11:25 AM Francisco Guerrero 
wrote:

> +1 (nb)
>
> On 2023/06/13 16:22:55 Andrés de la Peña wrote:
> > +1
> >
> > On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:
> >
> > > +1
> > > --
> > > *From:* David Capwell 
> > > *Sent:* Tuesday, June 13, 2023 8:37:10 AM
> > > *To:* dev 
> > > *Subject:* Re: [VOTE] CEP-8 Datastax Drivers Donation
> > >
> > > +1
> > >
> > > On Jun 13, 2023, at 7:59 AM, Josh McKenzie 
> wrote:
> > >
> > > +1
> > >
> > > On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
> > >
> > > +1 nb
> > >
> > > On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna <
> jeremy.hanna1...@gmail.com>
> > > wrote:
> > >
> > >
> > > Calling for a vote on CEP-8 [1].
> > >
> > > To clarify the intent, as Benjamin said in the discussion thread [2],
> the
> > > goal of this vote is simply to ensure that the community is in favor of
> > > the donation. Nothing more.
> > > The plan is to introduce the drivers, one by one. Each driver donation
> > > will need to be accepted first by the PMC members, as it is the case
> for
> > > any donation. Therefore the PMC should have full control on the pace at
> > > which new drivers are accepted.
> > >
> > > If this vote passes, we can start this process for the Java driver
> under
> > > the direction of the PMC.
> > >
> > > Jeremy
> > >
> > > 1.
> > >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> > > 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
> > >
> > >
> > >
> >
>

Re: [DISCUSS] Remove deprecated keyspace_count_warn_threshold and table_count_warn_threshold

[Andrés de la Peña]

Indeed "keyspace_count_warn_threshold" and "table_count_warn_threshold"
include system keyspaces and tables. Also, differently to the newer
guardrails, they are enabled by default.

I find that problematic because users need to know how many system
keyspaces/tables there are to know if they need to set the threshold value.
Moreover, if a new release adds some system tables, the threshold can start
to be triggered without changing the number of user tables. That would
force some users to update the threshold values during an upgrade. Even if
they are using the defaults. That situation would happen again in any
release adding new keyspaces/tables. I think adding new system tables is
not that uncommon, and indeed 5.0 does it.

I think that the combined decision of using a default value and counting
system tables was a mistake. If that's the case, I don't know for how long
we want to remain tied to that mistake. Especially when the old thresholds
tend to create upgrade issues on their own.

If we were going to use converters, we would need to know how many system
keyspaces/tables were on the version we are upgrading from. I don't know if
that information is available. Or perhaps we could assume that counting
system keyspaces/tables was a bug, and just translate changing the meaning
to not include them.





On Tue, 13 Jun 2023 at 16:51, David Capwell  wrote:

> > Have we been dropping support entirely for old params or using the
> @Replaces annotation into perpetuity?
>
>
> My understanding is that the goal is to keep things around in perpetuity
> unless it actively causes us harm… and with @Replaces, there tends to be no
> harm to keep around…
>
> Looking at
> https://github.com/apache/cassandra/commit/bae92ee139b411c94228f8fd5bb8befb4183ca9f
> we just marked them deprecated and created a brand new config that matched
> the old… which I feel was a bad idea…. Renaming configs are fine with
> @Replaces, but asking users to migrate with the idea of breaking them in
> the future is bad…
>
> The table_count_warn_threshold config is used at
> org.apache.cassandra.cql3.statements.schema.CreateTableStatement#clientWarnings
> The tables_warn_threshold config is used at
> org.apache.cassandra.cql3.statements.schema.CreateTableStatement#validate
>
> The only difference I see is that table_count_warn_threshold includes
> system tables where as tables_warn_threshold is only user tables…
>
> > I would like to propose removing the non-guardrail thresholds
> 'keyspace_count_warn_threshold' and 'table_count_warn_threshold'
> configuration settings on the trunk branch for the next major release.
>
> Deprecate in 4.1 is way too new for me to accept that, and its low effort
> to keep; breaking users is always a bad idea and doing it when not needed
> is bad…
>
> Honestly, I don’t see why we couldn’t use @Replaces here to solve the
> semantic gap… table_count_warn_threshold includes the system tables, so we
> just need a Converter that takes w/e the value the user put in and
> subtracts the system tables… which then gives us the user tables (matching
> tables_warn_threshold)
>
> > On Jun 13, 2023, at 7:57 AM, Josh McKenzie  wrote:
> >
> >> have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195
> when they were replaced/migrated to guardrails as part of CEP-3
> (Guardrails).
> > Have we been dropping support entirely for old params or using the
> @Replaces annotation into perpetuity?
> >
> > I dislike the idea of operators having to remember to update things
> between versions and being surprised when things change roughly equally to
> us carrying along undocumented deprecated param name mapping roughly
> equally. :)
> >
> > On Mon, Jun 12, 2023, at 5:56 PM, Dan Jatnieks wrote:
> >> Hello everyone,
> >>
> >> I would like to propose removing the non-guardrail thresholds
> 'keyspace_count_warn_threshold' and 'table_count_warn_threshold'
> configuration settings on the trunk branch for the next major release.
> >>
> >> These thresholds were first added with CASSANDRA-16309 in 4.0-beta4 and
> have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when
> they were replaced/migrated to guardrails as part of CEP-3 (Guardrails).
> >>
> >> I'd appreciate any thoughts about this. I will open a ticket to get
> started if there is support for doing this.
> >>
> >> Reference:
> >> https://issues.apache.org/jira/browse/CASSANDRA-16309
> >> https://issues.apache.org/jira/browse/CASSANDRA-17195
> >> CEP-3: Guardrails
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-3%3A+Guardrails
> >>
> >>
> >> Thanks,
> >> Dan Jatnieks
>
>
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Dinesh Joshi]

+1

Dinesh

> On Jun 13, 2023, at 7:15 AM, Jeremy Hanna  wrote:
> 
> 
> Calling for a vote on CEP-8 [1].
> 
> To clarify the intent, as Benjamin said in the discussion thread [2], the 
> goal of this vote is simply to ensure that the community is in favor of the 
> donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation will 
> need to be accepted first by the PMC members, as it is the case for any 
> donation. Therefore the PMC should have full control on the pace at which new 
> drivers are accepted.
> 
> If this vote passes, we can start this process for the Java driver under the 
> direction of the PMC.
> 
> Jeremy
> 
> 1. 
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Abe Ratnofsky]

+1 (nb)On Jun 13, 2023, at 09:23, Andrés de la Peña  wrote:+1On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:






+1






From: David Capwell 
Sent: Tuesday, June 13, 2023 8:37:10 AM
To: dev 
Subject: Re: [VOTE] CEP-8 Datastax Drivers Donation
 

+1


On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:



+1




On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:


+1 nb




On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna  wrote:






Calling for a vote on CEP-8 [1].



To clarify the intent, as Benjamin said in the discussion thread [2], the goal of this vote is simply to ensure that the community is in favor of the donation. Nothing more.

The plan is to introduce the drivers, one by one. Each driver donation will need to be accepted first by the PMC members, as it is the case for any donation. Therefore the PMC should have full control on the pace at which new drivers are
 accepted.



If this vote passes, we can start this process for the Java driver under the direction of the PMC.



Jeremy



1. https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation

2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

[DISCUSS] Remove org.apache.cassandra.io.sstable.SSTableHeaderFix in trunk (5.0)?

[David Capwell]

org.apache.cassandra.io.sstable.SSTableHeaderFix was added due to bugs in 3.6 
causing invalidate types or incompatible types (due to toString changes) in the 
SSTableHeader… this logic runs on start and rewrites all Stats files that had a 
mismatch from the local schema; with 5.0 requiring upgrades from 4.x only, this 
logic should have already run as its a 3.x to 4.0 migration step (though users 
are able to opt out [1]) which should have already fixed the SSTables to have 
correct schema…

Why is this a problem now?  CASSANDRA-18504 is adding a lot of property/fuzz 
tests to the type system and the read/write path, which has found several bugs; 
fixing some of the bugs actually impacts SSTableHeader because it requires 
generating and working with types that are not valid, so it can fix them…   By 
removing this logic, we can push this type validation into the type system to 
avoid generating incorrect types.  

If we wish to keep this class, we need to maintain allowing invalid types to be 
created, which may cause bugs down the road.


[1] if a user opts out there are 2 real cases that are impacted: UDTs, and 
collections of collections…
* For UDTs, the frozen vs non-frozen type are not the same, so mixing these 
causes us to fail to read the data, failing the read…. I believe 
writes/compactions will not corrupt the data, but anything that touches these 
SSTables will fail due to the schema mismatch… the only way to resolve this is 
to fix the SSTables… If you disabled in 4.x, you were living with broken / 
unreadable SSTables, so by removing 5.0 would loose the ability to repair them 
(but 4.x would still be able to)
* for collections of collections, this is less of an issue.  The logic would 
detect that the collection has a non-frozen collection as the element, so would 
migrate them to frozen.  This behavior has been moved to the type system, so a 
read from SSTable of “list>” automagically becomes a 
"ListType(FrozenType(ListType(Int32Type)))”.  The SSTables are not “fixed”, but 
compaction is able to read the data correctly, and the new SSTables will have 
the correct header.  

Re: [VOTE] CEP-8 Datastax Drivers Donation

[German Eichberger via dev]

+ 1

Great to see this moving forward!

From: Abe Ratnofsky 
Sent: Tuesday, June 13, 2023 10:09 AM
To: dev@cassandra.apache.org 
Subject: [EXTERNAL] Re: [VOTE] CEP-8 Datastax Drivers Donation

+1 (nb)

On Jun 13, 2023, at 09:23, Andrés de la Peña  wrote:


+1

On Tue, 13 Jun 2023 at 16:40, Yifan Cai 
mailto:yc25c...@gmail.com>> wrote:
+1

From: David Capwell mailto:dcapw...@apple.com>>
Sent: Tuesday, June 13, 2023 8:37:10 AM
To: dev mailto:dev@cassandra.apache.org>>
Subject: Re: [VOTE] CEP-8 Datastax Drivers Donation

+1

On Jun 13, 2023, at 7:59 AM, Josh McKenzie 
mailto:jmcken...@apache.org>> wrote:

+1

On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
+1 nb

On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
mailto:jeremy.hanna1...@gmail.com>> wrote:

Calling for a vote on CEP-8 [1].

To clarify the intent, as Benjamin said in the discussion thread [2], the goal 
of this vote is simply to ensure that the community is in favor of the 
donation. Nothing more.
The plan is to introduce the drivers, one by one. Each driver donation will 
need to be accepted first by the PMC members, as it is the case for any 
donation. Therefore the PMC should have full control on the pace at which new 
drivers are accepted.

If this vote passes, we can start this process for the Java driver under the 
direction of the PMC.

Jeremy

1. 
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [DISCUSS] Remove deprecated keyspace_count_warn_threshold and table_count_warn_threshold

[David Capwell]

> I think that the combined decision of using a default value and counting 
> system tables was a mistake

Default value I agree with you; features should be off by default!  If we 
remove the default then we disable the feature by default (which im cool with) 
and for anyone who changed the config, they would keep their behavior

As for system tables… each table adds a cost to our bookkeeping, so when we add 
new tables the cost grows and the memory per table decreases, does it not?  
Warning that too many tables (including system) may have negative behavior I 
think is fine, its only if we start to fail is when things become a problem 
(upgrading to 5.0 can’t happen due to too many tables added in the release?); 
think the feature was warn only, so that should be fine.  With that, I kinda 
don’t agree that including system tables is a mistake, as we add more we allow 
less for user tables before we start to have issues…. At the same time, if we 
have improvements in newer versions that allows higher number of tables, the 
user then has to update their configs (well, as long as we don’t make things 
worse a smaller limit than needed is fine…)

> we would need to know how many system keyspaces/tables were on the version we 
> are upgrading from

Do we?  The logic was pulling from local schema, so to keep the same behavior 
we would need to do the same; being version dependent would actually break the 
semantics as far as I can tell.

> On Jun 13, 2023, at 9:50 AM, Andrés de la Peña  wrote:
> 
> Indeed "keyspace_count_warn_threshold" and "table_count_warn_threshold" 
> include system keyspaces and tables. Also, differently to the newer 
> guardrails, they are enabled by default. 
> 
> I find that problematic because users need to know how many system 
> keyspaces/tables there are to know if they need to set the threshold value. 
> Moreover, if a new release adds some system tables, the threshold can start 
> to be triggered without changing the number of user tables. That would force 
> some users to update the threshold values during an upgrade. Even if they are 
> using the defaults. That situation would happen again in any release adding 
> new keyspaces/tables. I think adding new system tables is not that uncommon, 
> and indeed 5.0 does it.
> 
> I think that the combined decision of using a default value and counting 
> system tables was a mistake. If that's the case, I don't know for how long we 
> want to remain tied to that mistake. Especially when the old thresholds tend 
> to create upgrade issues on their own.
> 
> If we were going to use converters, we would need to know how many system 
> keyspaces/tables were on the version we are upgrading from. I don't know if 
> that information is available. Or perhaps we could assume that counting 
> system keyspaces/tables was a bug, and just translate changing the meaning to 
> not include them.
> 
> 
> 
> 
> 
> On Tue, 13 Jun 2023 at 16:51, David Capwell  > wrote:
>> > Have we been dropping support entirely for old params or using the 
>> > @Replaces annotation into perpetuity?
>> 
>> 
>> My understanding is that the goal is to keep things around in perpetuity 
>> unless it actively causes us harm… and with @Replaces, there tends to be no 
>> harm to keep around…
>> 
>> Looking at 
>> https://github.com/apache/cassandra/commit/bae92ee139b411c94228f8fd5bb8befb4183ca9f
>>  we just marked them deprecated and created a brand new config that matched 
>> the old… which I feel was a bad idea…. Renaming configs are fine with 
>> @Replaces, but asking users to migrate with the idea of breaking them in the 
>> future is bad…
>> 
>> The table_count_warn_threshold config is used at 
>> org.apache.cassandra.cql3.statements.schema.CreateTableStatement#clientWarnings
>> The tables_warn_threshold config is used at 
>> org.apache.cassandra.cql3.statements.schema.CreateTableStatement#validate
>> 
>> The only difference I see is that table_count_warn_threshold includes system 
>> tables where as tables_warn_threshold is only user tables…
>> 
>> > I would like to propose removing the non-guardrail thresholds 
>> > 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' 
>> > configuration settings on the trunk branch for the next major release.
>> 
>> Deprecate in 4.1 is way too new for me to accept that, and its low effort to 
>> keep; breaking users is always a bad idea and doing it when not needed is 
>> bad…
>> 
>> Honestly, I don’t see why we couldn’t use @Replaces here to solve the 
>> semantic gap… table_count_warn_threshold includes the system tables, so we 
>> just need a Converter that takes w/e the value the user put in and subtracts 
>> the system tables… which then gives us the user tables (matching 
>> tables_warn_threshold)
>> 
>> > On Jun 13, 2023, at 7:57 AM, Josh McKenzie > > > wrote:
>> > 
>> >> have subsequently been deprecated since 4.1-alpha in CASSANDRA-17195 when 
>> >> they were repl

Re: [VOTE] CEP-8 Datastax Drivers Donation

[C. Scott Andreas]

+1nbOn Jun 13, 2023, at 10:25 AM, German Eichberger via dev  wrote:






+ 1




Great to see this moving forward!


From: Abe Ratnofsky 
Sent: Tuesday, June 13, 2023 10:09 AM
To: dev@cassandra.apache.org 
Subject: [EXTERNAL] Re: [VOTE] CEP-8 Datastax Drivers Donation
 



+1 (nb)

On Jun 13, 2023, at 09:23, Andrés de la Peña  wrote:





+1



On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:





+1






From: David Capwell 
Sent: Tuesday, June 13, 2023 8:37:10 AM
To: dev 
Subject: Re: [VOTE] CEP-8 Datastax Drivers Donation
 

+1


On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:



+1




On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:


+1 nb




On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna  wrote:






Calling for a vote on CEP-8 [1].



To clarify the intent, as Benjamin said in the discussion thread [2], the goal of this vote is simply to ensure that the community is in favor of the donation. Nothing more.

The plan is to introduce the drivers, one by one. Each driver donation will need to be accepted first by the PMC members, as it is the case for any donation. Therefore the PMC should have full control on the pace at which new drivers are accepted.



If this vote passes, we can start this process for the Java driver under the direction of the PMC.



Jeremy



1. https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation

2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

[Mick Semb Wever]

> I want also to clarify and discuss with you some points related to the
> SystemPropertiesTable virtual table.
>
> 1. Hide non-production environment properties in the SystemProperties
> virtual table.
>
> As you may know, the result of the query on the virtual table
> currently includes the environment properties related to tests as well
> (used internally for our testing purposes) ~ 42 out of 290. This seems
> a bit redundant for production use and floods the query output with
> the things you don't need to think about. I think we can add a new
> property -Dcassandra.vt.show.test.system.properties (false by default)
> properties to hide these test-related properties for production
> environments while still using them for our test runs and build
> scripts. Hiding test properties is not a regression in this case. Any
> thoughts?
>
>
> 2. The absence of a description of the system properties.
>
> I have found that there is no good description of the system
> properties used to configure production environments. Is there any
> documentation for this case? Do we need to keep these pages up to
> date?



It's a yes to both (1) and (2) from me.

Appreciate it Maxim.  Also that CASSANDRA-18587 is taking into account
compatibility.

Re: [VOTE] CEP-8 Datastax Drivers Donation

[DuyHai Doan]

+1 nb

On Tue, Jun 13, 2023 at 8:00 PM C. Scott Andreas 
wrote:

> +1nb
>
> On Jun 13, 2023, at 10:25 AM, German Eichberger via dev <
> dev@cassandra.apache.org> wrote:
>
> 
> + 1
>
> Great to see this moving forward!
> --
> *From:* Abe Ratnofsky 
> *Sent:* Tuesday, June 13, 2023 10:09 AM
> *To:* dev@cassandra.apache.org 
> *Subject:* [EXTERNAL] Re: [VOTE] CEP-8 Datastax Drivers Donation
>
> +1 (nb)
>
> On Jun 13, 2023, at 09:23, Andrés de la Peña  wrote:
>
> 
> +1
>
> On Tue, 13 Jun 2023 at 16:40, Yifan Cai  wrote:
>
> +1
> --
> *From:* David Capwell 
> *Sent:* Tuesday, June 13, 2023 8:37:10 AM
> *To:* dev 
> *Subject:* Re: [VOTE] CEP-8 Datastax Drivers Donation
>
> +1
>
> On Jun 13, 2023, at 7:59 AM, Josh McKenzie  wrote:
>
> +1
>
> On Tue, Jun 13, 2023, at 10:55 AM, Jeremiah Jordan wrote:
>
> +1 nb
>
> On Jun 13, 2023 at 9:14:35 AM, Jeremy Hanna 
> wrote:
>
>
> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the
> goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
>
> Jeremy
>
> 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>
>
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Michael Shuler]

+1

On 6/13/23 09:14, Jeremy Hanna wrote:

Calling for a vote on CEP-8 [1].

To clarify the intent, as Benjamin said in the discussion thread [2], 
the goal of this vote is simply to ensure that the community is in favor 
of the donation. Nothing more.
The plan is to introduce the drivers, one by one. Each driver donation 
will need to be accepted first by the PMC members, as it is the case for 
any donation. Therefore the PMC should have full control on the pace at 
which new drivers are accepted.


If this vote passes, we can start this process for the Java driver under 
the direction of the PMC.


Jeremy

1. 
https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation 
2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp 

Re: [DISCUSS] Moving system property names to the CassandraRelevantProperties

[Ekaterina Dimitrova]

Thank you Maxim!

I also agree with 1 and 2. Also, thank you for considering documentation.
The datastax docs page mentioned here is not maintained by the project, the
other one can get some love. I guess if we are adding description column,
we can automate the way we populate that page. But I think this is
something for a separate ticket for docs mgmt.

On Tue, 13 Jun 2023 at 15:05, Mick Semb Wever  wrote:

> > I want also to clarify and discuss with you some points related to the
> > SystemPropertiesTable virtual table.
> >
> > 1. Hide non-production environment properties in the SystemProperties
> > virtual table.
> >
> > As you may know, the result of the query on the virtual table
> > currently includes the environment properties related to tests as well
> > (used internally for our testing purposes) ~ 42 out of 290. This seems
> > a bit redundant for production use and floods the query output with
> > the things you don't need to think about. I think we can add a new
> > property -Dcassandra.vt.show.test.system.properties (false by default)
> > properties to hide these test-related properties for production
> > environments while still using them for our test runs and build
> > scripts. Hiding test properties is not a regression in this case. Any
> > thoughts?
> >
> >
> > 2. The absence of a description of the system properties.
> >
> > I have found that there is no good description of the system
> > properties used to configure production environments. Is there any
> > documentation for this case? Do we need to keep these pages up to
> > date?
>
>
>
> It's a yes to both (1) and (2) from me.
>
> Appreciate it Maxim.  Also that CASSANDRA-18587 is taking into account
> compatibility.
>

Re: [DISCUSS] Remove deprecated keyspace_count_warn_threshold and table_count_warn_threshold

[Josh McKenzie]

> Warning that too many tables (including system) may have negative behavior I 
> think is fine
This reminds me of the current situation with our tests where we just keep 
adding more and more without really considering the value of the current set 
and the costs of that body of work as it keeps growing.

Having some kind of signal that we need to do some housekeeping with our system 
tables, or *something* in the feedback loop that helps us keep on top of this 
hygiene over time, seems like a clear benefit to me.

On Tue, Jun 13, 2023, at 1:42 PM, David Capwell wrote:
>> I think that the combined decision of using a default value and counting 
>> system tables was a mistake
> 
> Default value I agree with you; features should be off by default!  If we 
> remove the default then we disable the feature by default (which im cool 
> with) and for anyone who changed the config, they would keep their behavior
> 
> As for system tables… each table adds a cost to our bookkeeping, so when we 
> add new tables the cost grows and the memory per table decreases, does it 
> not?  Warning that too many tables (including system) may have negative 
> behavior I think is fine, its only if we start to fail is when things become 
> a problem (upgrading to 5.0 can’t happen due to too many tables added in the 
> release?); think the feature was warn only, so that should be fine.  With 
> that, I kinda don’t agree that including system tables is a mistake, as we 
> add more we allow less for user tables before we start to have issues…. At 
> the same time, if we have improvements in newer versions that allows higher 
> number of tables, the user then has to update their configs (well, as long as 
> we don’t make things worse a smaller limit than needed is fine…)
> 
>> we would need to know how many system keyspaces/tables were on the version 
>> we are upgrading from
> 
> Do we?  The logic was pulling from local schema, so to keep the same behavior 
> we would need to do the same; being version dependent would actually break 
> the semantics as far as I can tell.
> 
>> On Jun 13, 2023, at 9:50 AM, Andrés de la Peña  wrote:
>> 
>> Indeed "keyspace_count_warn_threshold" and "table_count_warn_threshold" 
>> include system keyspaces and tables. Also, differently to the newer 
>> guardrails, they are enabled by default. 
>> 
>> I find that problematic because users need to know how many system 
>> keyspaces/tables there are to know if they need to set the threshold value. 
>> Moreover, if a new release adds some system tables, the threshold can start 
>> to be triggered without changing the number of user tables. That would force 
>> some users to update the threshold values during an upgrade. Even if they 
>> are using the defaults. That situation would happen again in any release 
>> adding new keyspaces/tables. I think adding new system tables is not that 
>> uncommon, and indeed 5.0 does it.
>> 
>> I think that the combined decision of using a default value and counting 
>> system tables was a mistake. If that's the case, I don't know for how long 
>> we want to remain tied to that mistake. Especially when the old thresholds 
>> tend to create upgrade issues on their own.
>> 
>> If we were going to use converters, we would need to know how many system 
>> keyspaces/tables were on the version we are upgrading from. I don't know if 
>> that information is available. Or perhaps we could assume that counting 
>> system keyspaces/tables was a bug, and just translate changing the meaning 
>> to not include them.
>> 
>> 
>> 
>> 
>> 
>> On Tue, 13 Jun 2023 at 16:51, David Capwell  wrote:
>>> > Have we been dropping support entirely for old params or using the 
>>> > @Replaces annotation into perpetuity?
>>> 
>>> 
>>> My understanding is that the goal is to keep things around in perpetuity 
>>> unless it actively causes us harm… and with @Replaces, there tends to be no 
>>> harm to keep around…
>>> 
>>> Looking at 
>>> https://github.com/apache/cassandra/commit/bae92ee139b411c94228f8fd5bb8befb4183ca9f
>>>  we just marked them deprecated and created a brand new config that matched 
>>> the old… which I feel was a bad idea…. Renaming configs are fine with 
>>> @Replaces, but asking users to migrate with the idea of breaking them in 
>>> the future is bad…
>>> 
>>> The table_count_warn_threshold config is used at 
>>> org.apache.cassandra.cql3.statements.schema.CreateTableStatement#clientWarnings
>>> The tables_warn_threshold config is used at 
>>> org.apache.cassandra.cql3.statements.schema.CreateTableStatement#validate
>>> 
>>> The only difference I see is that table_count_warn_threshold includes 
>>> system tables where as tables_warn_threshold is only user tables…
>>> 
>>> > I would like to propose removing the non-guardrail thresholds 
>>> > 'keyspace_count_warn_threshold' and 'table_count_warn_threshold' 
>>> > configuration settings on the trunk branch for the next major release.
>>> 
>>> Deprecate in 4.1 is way too new for me to ac

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Derek Chen-Becker]

+1

On Tue, Jun 13, 2023 at 8:15 AM Jeremy Hanna 
wrote:

> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the
> goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
>
> Jeremy
>
> 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>


-- 
+---+
| Derek Chen-Becker |
| GPG Key available at https://keybase.io/dchenbecker and   |
| https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org |
| Fngrprnt: EB8A 6480 F0A3 C8EB C1E7  7F42 AFC5 AFEE 96E4 6ACC  |
+---+

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Nate McCall]

+1

On Wed, Jun 14, 2023 at 2:15 AM Jeremy Hanna 
wrote:

> Calling for a vote on CEP-8 [1].
>
> To clarify the intent, as Benjamin said in the discussion thread [2], the
> goal of this vote is simply to ensure that the community is in favor of
> the donation. Nothing more.
> The plan is to introduce the drivers, one by one. Each driver donation
> will need to be accepted first by the PMC members, as it is the case for
> any donation. Therefore the PMC should have full control on the pace at
> which new drivers are accepted.
>
> If this vote passes, we can start this process for the Java driver under
> the direction of the PMC.
>
> Jeremy
>
> 1.
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>

Re: [VOTE] CEP-8 Datastax Drivers Donation

[Jake Luciani]

+1

On Tue, Jun 13, 2023 at 7:14 PM Nate McCall  wrote:

> +1
>
> On Wed, Jun 14, 2023 at 2:15 AM Jeremy Hanna 
> wrote:
>
>> Calling for a vote on CEP-8 [1].
>>
>> To clarify the intent, as Benjamin said in the discussion thread [2], the
>> goal of this vote is simply to ensure that the community is in favor of
>> the donation. Nothing more.
>> The plan is to introduce the drivers, one by one. Each driver donation
>> will need to be accepted first by the PMC members, as it is the case for
>> any donation. Therefore the PMC should have full control on the pace at
>> which new drivers are accepted.
>>
>> If this vote passes, we can start this process for the Java driver under
>> the direction of the PMC.
>>
>> Jeremy
>>
>> 1.
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>>
>

-- 
http://twitter.com/tjake

Re: [VOTE] CEP-8 Datastax Drivers Donation

[guo Maxwell]

+1

Jake Luciani  于2023年6月14日周三 09:21写道：

> +1
>
> On Tue, Jun 13, 2023 at 7:14 PM Nate McCall  wrote:
>
>> +1
>>
>> On Wed, Jun 14, 2023 at 2:15 AM Jeremy Hanna 
>> wrote:
>>
>>> Calling for a vote on CEP-8 [1].
>>>
>>> To clarify the intent, as Benjamin said in the discussion thread [2],
>>> the goal of this vote is simply to ensure that the community is in
>>> favor of the donation. Nothing more.
>>> The plan is to introduce the drivers, one by one. Each driver donation
>>> will need to be accepted first by the PMC members, as it is the case for
>>> any donation. Therefore the PMC should have full control on the pace at
>>> which new drivers are accepted.
>>>
>>> If this vote passes, we can start this process for the Java driver under
>>> the direction of the PMC.
>>>
>>> Jeremy
>>>
>>> 1.
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>>>
>>
>
> --
> http://twitter.com/tjake
>


-- 
you are the apple of my eye !