At this point you have two options: 1. Create a different LogOutput on the central server for each output file that you want to generate, and give each of them a message_matcher such that they only catch the entries that are specific to the file(s) they should be catching. LogstreamerInput will set the `Logger` value uniquely for each separate input plugin (or for each separate Logstream, if a single input plugin is generating multiple streams) so matching on the Logger value is probably what you'd want. Note that this assumes that you're using LogstreamerInput correctly in that you're not interleaving unrelated log files into a single stream.
2. Alternatively, you could use a SandboxOutput, which would let you write a bit of Lua code to do whatever you want. This would give you a lot more flexibility. You'd only need a single output, which would look at each message coming in and make a decision re: where to write it out to, even generating new files on the fly if new message types start coming in. Hope this helps, -r On 05/22/2016 07:45 PM, Watson, Chris (DPIPWE) wrote:
HI all I have been using heka as a proof of concept on remote central logging at my office, everything is working great so far, but I am a little confused on the next step. Currently I have been using TCPOutput to write to a remote server any and all logs in directories to a single file, for each server we have EG: On output box: [Accesslogs_apache] type = "LogstreamerInput" log_directory = "/var/log/apache2" file_match = '(?P<FileName>[^/]+).log' [LogOutput] message_matcher = "TRUE" encoder = "PayloadEncoder" [server1_output] type = "TcpOutput" address = "central-server-IP:5565" local_address = "Local-IP:5565" message_matcher = "TRUE" Then on central-server: [apache_server1] type = "FileOutput" message_matcher = "Hostname == 'server1' || Hostname == 'server1-IP'" path = "/logs/production/apache2/server1-%Y%m%d.log" perm = "666" encoder = "PayloadEncoder" rotation_interval = 24 that all works fine. My issue is I now need to produce multiple log files from the same server and have them output to different files as dumping multiple logs from server1 into one large file on the central server is no longer practical, but I am having a really hard time getting this working EG: Server1 has two different logs files: ‘foo.log’ and ‘bar.log’ I want both these logs to be passed to central-server and both of them to create different FileOutput’s to separate outputs so they appear on the Central server: /logs/production/apache2/foo.log /logs/production/apache2/bar.log Currently they all being dumped to a single file "/logs/production/apache2/server1-%Y%m%d.log" Where do I need to make the definition that these are two separate files, on the central server or on server1. ------------------------------------------------------------------------ CONFIDENTIALITY NOTICE AND DISCLAIMER The information in this transmission may be confidential and/or protected by legal professional privilege, and is intended only for the person or persons to whom it is addressed. If you are not such a person, you are warned that any disclosure, copying or dissemination of the information is unauthorised. If you have received the transmission in error, please immediately contact this office by telephone, fax or email, to inform us of the error and to enable arrangements to be made for the destruction of the transmission, or its return at our cost. No liability is accepted for any unauthorised use of the information contained in this transmission. _______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
_______________________________________________ Heka mailing list [email protected] https://mail.mozilla.org/listinfo/heka
