Sure Chris. I would be watching for that ticket's closure. You had pointed
out about this ticket already and what I had decided was, while this gets
merged etc, I can work with the folks here and get a design out.
Thanks!
Sagar.
On Tue, Apr 9, 2019 at 9:58 PM Chris Lohfink wrote:
> You should r
You should really build it on/wait for
https://issues.apache.org/jira/browse/CASSANDRA-14629 (Can use a reviewer
please btw!) then you can provide each entry of the audit log binlog as a
partition in iterator format. Can use settings to find where binlog output
is and go from there.
Chris
On Tue,
Thanks Jon.
So, it will basically have to read the audit log file which can certainly
be implemented. One of the things to consider is, if for example, we have
some external system which is polling the audit log table on a given node
every minute to get incremental audit logs. In that case, it wil
The virtual table could read the data out of the audit log, just like
it could read a hosts file or list the output of the ps command.
On Wed, Apr 3, 2019 at 8:02 PM Sagar wrote:
>
> Thanks Alex.
>
> I was going through the implementation of Virtual tables thus far and the
> data that we get whe
Thanks Alex.
I was going through the implementation of Virtual tables thus far and the
data that we get when we query against them seems to be more point in time
like CachesTable or fairly static like Settings. Having said that, audit
log's nature of data doesn't fall in either of the 2 categories
Hi Sagar
3.x/4.x are versions for open source variant of drivers, while DSE versions
are 1.x/2.x
Description of this function is a
thttps://docs.datastax.com/en/drivers/java/3.6/
Sagar at "Tue, 26 Mar 2019 22:12:56 +0530" wrote:
S> Thanks Andy,
S> This enhancement is in the datastax version
Thanks Andy,
This enhancement is in the datastax version and not in the apache cassandra
driver?
Thanks!
Sagar.
On Tue, Mar 26, 2019 at 3:23 AM Andy Tolbert
wrote:
> Hello
>
> 1) yes its local only. The driver by default does connect to each host
> > though so its pretty trivial to have a load
Hello
1) yes its local only. The driver by default does connect to each host
> though so its pretty trivial to have a load balancing policy that you can
> direct to specific hosts (this should probably be in the driver so people
> dont have to keep reimplementing it).
>
The capability to target a
Thanks Chris. I got caught up with a few things and couldn't reply back.
So, I re-looked this again and I think virtual tables can be used for audit
logging. Considering that they don't have any replication - so we won't be
clogging the network with replication IO.
In terms of storage, from what I
1) yes its local only. The driver by default does connect to each host
though so its pretty trivial to have a load balancing policy that you can
direct to specific hosts (this should probably be in the driver so people
dont have to keep reimplementing it).
2) yes, easiest way is to setup a whiteli
So, I went through the ticket for the creation of Virtual Tables(must say
it was quite a long ticket spanning across 4 years).
I see that there are a few tables created in the db.virtual package. These
appear to be metrics related tables.
Couple of questions here:
1) Do all the tables pertain on
Right, Thanks Jonathan and Chris.
Mean while, I would go through the 2 jira items to try and understand about
virtual tables.
Thanks!
Sagar.
On Tue, Mar 5, 2019 at 1:14 AM Jonathan Haddad wrote:
> Sagar,
>
> There isn't going to be much in the way of docs, since it's brand new and
> not really
Sagar,
There isn't going to be much in the way of docs, since it's brand new and
not really a public facing thing yet. As Chris pointed out, there's other
work that would need to be done to work on virtual tables for large
datasets.
Jon
On Mon, Mar 4, 2019 at 6:42 AM Chris Lohfink wrote:
> Wh
While you probably could put a virtual table wrapper over the binlogs, you
would want to wait for something like
https://issues.apache.org/jira/browse/CASSANDRA-14629 to get in so you
would not OOM. The current virtual table implementation requires you have
the entire result set to be returned at o
Hi Jonathan,
I couldn't find much literature on Virtual tables apart from this ticket:
https://issues.apache.org/jira/browse/CASSANDRA-7622
Any insights would be helpful.
Thanks!
Sagar.
On Sat, Mar 2, 2019 at 7:23 AM Jonathan Haddad wrote:
> Instead of logging to tables, putting a virtual ta
Instead of logging to tables, putting a virtual table around the audit /
query logs might be an option. Same with the commit log for cdc
On Fri, Mar 1, 2019 at 5:25 PM Sagar wrote:
> Thanks all for the pointers. Really insightful.
>
> Subroto I think that’s part of the enterprise version but yea
Thanks all for the pointers. Really insightful.
Subroto I think that’s part of the enterprise version but yeah even I have
seen it. Again not sure of the performance implications.
Sagar.
On Sat, 2 Mar 2019 at 5:15 AM, Subroto Barua
wrote:
> Datastax version has an option to store audit info to
Datastax version has an option to store audit info to dse_audit.audit_log
table; I do not know the performance impact since I use the file option
Subroto
> On Mar 1, 2019, at 9:40 AM, Jeremiah D Jordan
> wrote:
>
> AFAIK the Full Query Logging binary format was already made more general in
AFAIK the Full Query Logging binary format was already made more general in
order to support using that format for the audit logging.
-Jeremiah
> On Mar 1, 2019, at 11:38 AM, Joshua McKenzie wrote:
>
> Is there a world in which a general purpose, side-channel file storage
> format for transien
Is there a world in which a general purpose, side-channel file storage
format for transient things like this (hints, batches, audit logs, etc)
could be useful as a first class citizen in the codebase? i.e. a world in
which we refactored some of the hints-specific reader/writer code to be
used for t
Agreed with Dinesh and Josh. I would *never* put the audit log back in
Cassandra.
This is extendable, Sagar, so you're free to do as you want, but I'm very
opposed to putting a ticking time bomb in Cassandra proper.
Jon
On Thu, Feb 28, 2019 at 8:38 AM Dinesh Joshi
wrote:
> I strongly echo Jo
I strongly echo Josh’s sentiment. Imagine losing audit entries because C* is
overloaded? It’s fine if you don’t care about losing audit entries.
Dinesh
> On Feb 28, 2019, at 6:41 AM, Joshua McKenzie wrote:
>
> One of the things we've run into historically, on a *lot* of axes, is that
> "just p
One of the things we've run into historically, on a *lot* of axes, is that
"just put it in C*" for various functionality looks great from a user and
usability perspective, and proves to be something of a nightmare from an
admin / cluster behavior perspective.
i.e. - cluster suffering so you're wri
Thanks all for the pointers.
@Joseph,
I have gone through the links shared by you. Also, I have been looking at
the code base.
I understand the fact that pushing the logs to ES or Solr is a lot easier
to do. Having said that, the only reason I thought having something like
this might help is, if
> On Feb 27, 2019, at 10:41 AM, Joseph Lynch wrote:
>
> Vinay can confirm, but as far as I am aware we have no current plans to
> implement audit logging to a table directly, but the implementation is
> fully pluggable (like compaction, compression, etc ...). Check out the blog
> post [1] and
Hi Sagar,
Vinay can confirm, but as far as I am aware we have no current plans to
implement audit logging to a table directly, but the implementation is
fully pluggable (like compaction, compression, etc ...). Check out the blog
post [1] and documentation [2] Vinay wrote for more details, but the
I understand why you’d want it but it would add more data management to the
database. Generally for logging you could consider putting into ELK and then it
can be more queried on arbitrarily.
On Feb 27, 2019, 12:42 PM -0500, Dinesh Joshi ,
wrote:
> I don’t believe there is a plan to do it. If it
I don’t believe there is a plan to do it. If it were available in a table how
would that help you?
Dinesh
> On Feb 27, 2019, at 9:32 AM, Sagar wrote:
>
> Hey All,
>
> While following some of the recent developments on Cassandra, I found the
> new feature on Audit logging quite useful.
>
> I
Hey All,
While following some of the recent developments on Cassandra, I found the
new feature on Audit logging quite useful.
I wanted to understand is there any plan of pushing the audit logs to a
table?
Thanks!
Sagar.
29 matches
Mail list logo
