Log4j has all the necessary plugins to support numerous scenarios, and I've found three ways to support Graylog, for example:
1. GelfLayout + SocketAppender: send log messages straight to a Graylog server. Simplest setup (no additional dependencies required), but not the most reliable in theory. 2. GelfLayout + KafkaAppender: send log messages to Kafka first, then have a consumer on the other side ingest those into Graylog. This style is more flexible since the layout doesn't necessarily need to be a GelfLayout, but this is the most efficient way to handle that scenario. The cons, however, is that a message in memory that hasn't been sent to Kafka yet can be lost in a crash scenario, similar to the SocketAppender limitation. 3. GelfLayout + FlumeAppender: send messages formatted for Graylog, but persist locally first before passing off to another flume agent. I'm not too familiar with this setup, but based on what Ralph has explained before, this is probably the most reliable way to ensure logs are aggregated. Similar patterns can be followed for other services like Logstash, Splunk, etc. Anyways, what I'm looking for here are suggestions on architecture here and perhaps the desire to get a page written in the manual describing these types of distributed system logging scenarios. If we can explain how to follow these architectures, we can also find any inefficiencies in them to improve on. -- Matt Sicker <boa...@gmail.com>
