Thank you!
In fact I modified nginx_access decoder, used gsub to clean the payload and it
works.
I will try out using token splitter.
------------------ Original ------------------
From: "Rob Miller"<[email protected]>;
Date: Fri, Jun 3, 2016 02:46 AM
To: "张晔"<[email protected]>;
Cc: "heka"<[email protected]>;
Subject: Re: [heka] heka has trouble receiving remote openresty log
LogstreamerInput will default to using a TokenSplitter setup which uses newline
as the record boundary. UdpInput by default uses NullSplitter, which means that
the contents of the entire UDP packet will be assumed to be a single record. I
have no idea whether openresty sends multiple log lines per UDP packet, but if
it does that would likely cause what you're seeing. Try setting `splitter =
"TokenSplitter"` in your UdpInput and see if that helps.
If not you can try to debug by removing the decoder setting from your UdpInput.
Then your LogOutput should get the raw output that's arriving on the UDP port
as the payload of each message.
-r
On 06/01/2016 09:07 PM, 张晔 wrote:
> Hello,
>
> I'm collecting openresty log from serveral remote server.
>
> Here is my openresty config for sending log to my heka server:
>
> access_log syslog:server=10.100.0.117:10087 eru-lb;
>
> And here is my hekad config to handle these logs:
>
> [erulb_test_input]
> type = "UdpInput"
> address = "0.0.0.0:10087"
> decoder = "erulb_test_decoder"
>
> [erulb_test_decoder]
> type = "SandboxDecoder"
> filename = "lua_decoders/nginx_access.lua"
>
> [LBlogInput]
> type = "LogstreamerInput"
> log_directory = "/tmp/"
> file_match = 'elb.log'
> decoder = "erulb_test_decoder"
>
> [erulb_test_decoder.config]
> type = "combined"
> user_agent_transform = true
> log_format = '$host $remote_addr [$time_local] "$request" $status
> $body_bytes_sent "$http_referer" "$http_user_agent" $request_time
> $upstream_response_time $proxy_add_x_forwarded_for'
>
> [PayloadEncoder]
> [RstEncoder]
> [LogOutput]
> message_matcher = "Logger != 'hekad'"
> encoder = "RstEncoder"
>
> At first, there is no output for my openresty log. Then I changed my
> openresty config into:
>
> access_log /tmp/elb.log eru-lb;
>
> Writing logs to local file and let heka read local file. And heka did
> give output expected:
>
> So, Why heka can't handle openresty logs from remote servers? And how to
> deal with it?
>
> Cheers,
> Dantepy
>
>
>
> _______________________________________________
> Heka mailing list
> [email protected]
> https://mail.mozilla.org/listinfo/heka
>
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
_______________________________________________
Heka mailing list
[email protected]
https://mail.mozilla.org/listinfo/heka
