zhu.chenxiang created FLUME-3019:
------------------------------------
Summary: AvroSource throws OutOfMemory error when building an
elastic client
Key: FLUME-3019
URL: https://issues.apache.org/jira/browse/FLUME-3019
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.7.0
Environment: Linux version 4.1.7-15.23.amzn1.x86_64
Reporter: zhu.chenxiang
I use two AvroSource nodes to receive data,The client use AvroSink on aws
elastic ec2.
I find that when Client in case of power failure or broken network，the netty
server will not release the resource.
so when my client is elastic,the server still keep a lot of channels.After a
long time running ,throws OutOfMemory error.
I try to add IdleStateHandler in AdvancedChannelPipelineFactory，but allChannels
is a private variable in org.apache.avro.ipc.NettyServer，I cannot remove
Channel from allChannels.
org.jboss.netty.util.Timer timer = new HashedWheelTimer();
pipeline.addLast("timeout", new IdleStateHandler(timer, 300, 0, 0));
pipeline.addLast("hearbeat", new Heartbeat());
private class Heartbeat extends IdleStateAwareChannelHandler {
@Override
public void channelIdle(ChannelHandlerContext ctx, IdleStateEvent e)
throws Exception {
super.channelIdle(ctx, e);
if(e.getState() == IdleState.WRITER_IDLE){
e.getChannel().close();
//org.apache.avro.ipc.NettyServer allChannels
//allChannels.remove(e.getChannel());
logger.info("Connection to {} closed.",
e.getChannel().getRemoteAddress());
}
}
}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
