Re: [general] Recommended event log storage?

2021-08-04 Thread Ralph Goers
Actually, the major reason for the Flume Appender is for audit logging. It provides true guaranteed delivery. So once the appender returns your can be sure the event will eventually make it to its end destination. Ralph > On Aug 4, 2021, at 1:04 AM, Volkan Yazıcı wrote: > > We have our Redi

Re: [general] Recommended event log storage?

2021-08-04 Thread Matt Sicker
My current setup is essentially a small modification to the EcsLayout in JsonTemplateLayout (to ignore certain MDC values) to a rolling file appender with a sidecar container for forwarding log files to Splunk. The event log I'm considering here does not directly involve finances (you could stretch

Re: [general] Recommended event log storage?

2021-08-04 Thread Dominik Psenner
Hi I like Json as the transport format over mqtt. A few hundred lines of code persist events to a postgresql database by calling a stored procedure that handles the json directly. The stored procedure allows for plugging in some fancy aggregation logic or distribution across tables or even retenti

Re: [general] Recommended event log storage?

2021-08-04 Thread Volkan Yazıcı
We have our Redis-shielded ELK stack – Redis acts as a buffer for fluctuating Elasticsearch ingestion performance and downtime. Apps use log4j2-redis-appender , not the official Log4j one. This said, I am lobbying for replacing Redis with Google Cloud Pu