[DISCUSS] Diagnostic events in virtual tables

[Stefan Miklosovic]

Hi,

I wonder if people would be interested in having diagnostic events in
virtual tables?

I put something together here (heavy wip) (1) but that is the idea I have.

I know that Stefan Podkowinski did a spectacular job under
CASSANDRA-12944 and the possibility to persist locally was elaborated
on in (2) where the conclusion was made that maybe it is more suitable
to put it into chronicle queues via BinLog path and so on.

The benefits of bin log is that we have events persisted and they can
be inspected further when node is offline.

While data in virtual tables cease to exist after nodes are down, one
nice benefit of having it in virtual tables is that we can query it
comfortably via CQL and I think that this solution is more suitable to
have on an every day basis from operator's point of view. There is
still a way to dump it somewhere else anyway if one is really
interested in doing so.

Do you think that the bin log solution is overall superior to the
virtual tables approach and we should just forget about having it in
virtual tables?

If that is the case, what I would like to see is to have some
pluggability here so I might implement this on my own and configure a
node to put it to virtual tables for me, it does not necessarily have
to be the part of Cassandra code base if we are strongly against that.

(1) 
https://github.com/instaclustr/cassandra/commit/0dd60dc0a847619fdb704b700154e624b21a0c35
(2) https://issues.apache.org/jira/browse/CASSANDRA-13460

Regards

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [DISCUSS] Virtual Tables and the future of NodeTool/JMX

[Chris Lohfink]

> a) Allow VirtualTables to be settable - to support changing parameters (ie.
> nodetool setcompactionthroughput 32).


Can be accomplished by updating or inserting on the settings virtual table
via

UPDATE system_views.settings SET value = 32 WHERE name =
'compaction_throughput';
INSERT INTO system_views.settings (name, value) VALUES
('compaction_throughput', 32);

It also aligns configuration in yaml names with what you use to update it
at runtime, vs having different naming and one off commands for each
individual setting. That was pushed off in the settings virtual table for
post 4.0 however. (see original patch in
https://issues.apache.org/jira/browse/CASSANDRA-14573)

There is a long way to deprecate JMX though, a lot of things are absolutely
necessary before even considering it like
https://issues.apache.org/jira/browse/CASSANDRA-14629

Chris

On Fri, Jul 16, 2021 at 11:39 AM Aleksey Yeschenko 
wrote:

> I would say just go with it. JMX isn’t quite deprecated yet, and if we
> ever even end up doing that, it’s not going to be any time soon.
>
> > On 16 Jul 2021, at 13:32, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
> >
> > Thanks Benjamin for the understanding, but in the end, let's put aside
> > the frustration here, I think I can just kind of detach from that.
> >
> > However, in this particular case, I really think we should just finish
> > this and merge it and move on. By not doing so and waiting for the CEP
> > around this, we are just opening ourselves to a possibility that it
> > will not be finished / implemented and we will live without that
> > command.
> >
> > In other words, I just do not like to not do something, when it just
> > lies in front of me, just because of some assumption we make that
> > something will happen in the future. I do not get this approach. The
> > future might just change. Plus this is just an easy patch, no big
> > deal, it is not like I would have to revert the heaps of code to
> > accommodate future features.
> >
> > If I do not merge it, I am afraid this might happen:
> >
> > A user (lets call him Tommy), is excited about 4.0, they are using
> > some custom auditing and he wants to give a shot to native stuff we
> > provide here. So he goes and downloads, tries it, all is good.
> > Then he realises that it was a long time ago he set this up, messed
> > with that a bit and kept it running. But after a while, he forgot how
> > he set it up. So he looks into nodetool and sees, yay, enableauditlog,
> > disableauditlog, there are a bunch of get* commands too, but no status
> > of audit log? How can I know how that is set up?
> >
> > So Tommy reaches the community and says that hey, you are missing that
> > command, am I going to see it in 4.1 or in some patch release at least
> > to get it sooner?
> >
> > And we reply:
> >
> > Dear Tommy,
> >
> > we know it is missing, but wait! There is this CEP we are working day
> > and night on, it will solve the world hunger and it will be the best
> > thing since the sliced bread, you just have to wait a little bit
> > longer, because there are other things going on right now and it seems
> > to take longer than expected, lot of problems on the way, so maybe
> > return in 4.3 and there you get it.
> >
> > But Tommy does not care. He truly does not. He just wants to query the
> > state of the audit log by any means necessary. And once this happens
> > multiple times he just goes away.
> >
> > By the way, I really think this is a little bit more complicated than
> > it seems. It is hard to guess how the implementation will look like
> > but in our original discussion with Paulo, we were thinking about
> > having a completely separate repository for this where only the client
> > would be. There are few benefits - it might be just released
> > independently, if we truly separate this and we talk only CQL, I do
> > not see any reason why we should depend on the main Cassandra
> > repository. It also lowers the barrier for other people who would want
> > to implement the command they miss. However, I get that while we want
> > to support both approaches, it will be done and probably it has to be
> > done together, but I am afraid that we will just introduce a hybrid
> > which would take ages to finish fully so we can announce JMX
> > deprecated. There are a lot of commands to be migrated as well and
> > having a separate repository would at least make the main repo less
> > noisy. But since I am not fully aware of what approach we end up with
> > and how the implementation would look like in the end, I can not fully
> > advocate the separate repository approach because it might be just
> > rendered ineffective and wrong.
> >
> > That's just my rambling here but I would just close the issue with
> > 16725 by merging that and then we can fully focus on CEP and all
> > things related.
> >
> > Sounds good to everybody?
> >
> > Regards
> >
> >
> > On Fri, 16 Jul 2021 at 10:44, Benjamin Lerer  wrote:
> >>
> >> Thanks

Re: [DISCUSS] Virtual Tables and the future of NodeTool/JMX

[Paulo Motta]

> Can be accomplished by updating or inserting on the settings virtual
table via  UPDATE system_views.settings SET value = 32 WHERE name =
'compaction_throughput';

Oh nice, I wasn't aware this was available already, this is a great feature
to look forward to in 4.0!

> There is a long way to deprecate JMX though, a lot of things are
absolutely necessary before even considering it like
https://issues.apache.org/jira/browse/CASSANDRA-14629

Thanks for the pointer Chris. Will put together a proposal based on input
from this thread with a long term plan to slowly transition from JMX to
virtual tables.


Em seg., 19 de jul. de 2021 às 12:33, Chris Lohfink 
escreveu:

> > a) Allow VirtualTables to be settable - to support changing parameters
> (ie.
> > nodetool setcompactionthroughput 32).
>
>
> Can be accomplished by updating or inserting on the settings virtual table
> via
>
> UPDATE system_views.settings SET value = 32 WHERE name =
> 'compaction_throughput';
> INSERT INTO system_views.settings (name, value) VALUES
> ('compaction_throughput', 32);
>
> It also aligns configuration in yaml names with what you use to update it
> at runtime, vs having different naming and one off commands for each
> individual setting. That was pushed off in the settings virtual table for
> post 4.0 however. (see original patch in
> https://issues.apache.org/jira/browse/CASSANDRA-14573)
>
> There is a long way to deprecate JMX though, a lot of things are absolutely
> necessary before even considering it like
> https://issues.apache.org/jira/browse/CASSANDRA-14629
>
> Chris
>
> On Fri, Jul 16, 2021 at 11:39 AM Aleksey Yeschenko 
> wrote:
>
> > I would say just go with it. JMX isn’t quite deprecated yet, and if we
> > ever even end up doing that, it’s not going to be any time soon.
> >
> > > On 16 Jul 2021, at 13:32, Stefan Miklosovic <
> > stefan.mikloso...@instaclustr.com> wrote:
> > >
> > > Thanks Benjamin for the understanding, but in the end, let's put aside
> > > the frustration here, I think I can just kind of detach from that.
> > >
> > > However, in this particular case, I really think we should just finish
> > > this and merge it and move on. By not doing so and waiting for the CEP
> > > around this, we are just opening ourselves to a possibility that it
> > > will not be finished / implemented and we will live without that
> > > command.
> > >
> > > In other words, I just do not like to not do something, when it just
> > > lies in front of me, just because of some assumption we make that
> > > something will happen in the future. I do not get this approach. The
> > > future might just change. Plus this is just an easy patch, no big
> > > deal, it is not like I would have to revert the heaps of code to
> > > accommodate future features.
> > >
> > > If I do not merge it, I am afraid this might happen:
> > >
> > > A user (lets call him Tommy), is excited about 4.0, they are using
> > > some custom auditing and he wants to give a shot to native stuff we
> > > provide here. So he goes and downloads, tries it, all is good.
> > > Then he realises that it was a long time ago he set this up, messed
> > > with that a bit and kept it running. But after a while, he forgot how
> > > he set it up. So he looks into nodetool and sees, yay, enableauditlog,
> > > disableauditlog, there are a bunch of get* commands too, but no status
> > > of audit log? How can I know how that is set up?
> > >
> > > So Tommy reaches the community and says that hey, you are missing that
> > > command, am I going to see it in 4.1 or in some patch release at least
> > > to get it sooner?
> > >
> > > And we reply:
> > >
> > > Dear Tommy,
> > >
> > > we know it is missing, but wait! There is this CEP we are working day
> > > and night on, it will solve the world hunger and it will be the best
> > > thing since the sliced bread, you just have to wait a little bit
> > > longer, because there are other things going on right now and it seems
> > > to take longer than expected, lot of problems on the way, so maybe
> > > return in 4.3 and there you get it.
> > >
> > > But Tommy does not care. He truly does not. He just wants to query the
> > > state of the audit log by any means necessary. And once this happens
> > > multiple times he just goes away.
> > >
> > > By the way, I really think this is a little bit more complicated than
> > > it seems. It is hard to guess how the implementation will look like
> > > but in our original discussion with Paulo, we were thinking about
> > > having a completely separate repository for this where only the client
> > > would be. There are few benefits - it might be just released
> > > independently, if we truly separate this and we talk only CQL, I do
> > > not see any reason why we should depend on the main Cassandra
> > > repository. It also lowers the barrier for other people who would want
> > > to implement the command they miss. However, I get that while we want
> > > to