[
https://issues.apache.org/jira/browse/KAFKA-5861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162654#comment-16162654
]
Seweryn Habdank-Wojewodzki edited comment on KAFKA-5861 at 9/12/17 8:24 AM:
----------------------------------------------------------------------------
Property: num.stream.threads=4;
number of partitions in the topic = 10;
replication factor = 2
I see the following, when I do not send message at all, I can stop the app by
using kill PID.
When I am sending messages, kill PID does not work at all. I need to use kill
-9 PID.
To be precise, I had made shutdown hook and then I see log "Got shutdown
request ..." and nothing more.
If I will send single message and then wait quite a long time (more than 30 or
60 sec) then it is probable kill PID will work.
When I am sending steady couple (like 2-5) messages per second mostly (>90%)
kill PID does not work - I need to use kill -9 PID.
When I am sending like 1000 or more messages per second, then kill PID does not
work at all - again kill -9 PID was doing the job.
But I suppose kill -9 PID is not the way to go, or?
Core of my shutdown hook is below. The small comment - it is not important if I
have System.exit(0) or not, as the procedure never comes to this point :-).
Runtime.getRuntime().addShutdownHook( new Thread( () -> {
LOG.info( "Got shutdown requests ..." );
synchronized ( this ) {
streams.close( 5L, TimeUnit.SECONDS );
}
LOG.info( "Streamer is stopped." );
System.exit( 0 );
}
) );
was (Author: habdank):
Property: num.stream.threads=4;
number of partitions in the topic = 10;
replication factor = 2
I see the following, when I do not send message at all, I can stop the app by
using kill PID.
When I am sending messages, kill PID does not work at all. I need to use kill
-9 PID.
To be precise, I had made shutdown hook and then I see log "Got shutdown
request ..." and nothing more.
If I will send single message and then wait quite a long time (more than 30 or
60 sec) then it is probable kill PID will work.
When I am sending steady couple like 2-5 messages per second mostly (>90%) kill
PID does not work - I need to use kill -9 PID.
When I am sending like 1000 or more messages per second, then kill PID does not
work at all - again kill -9 PID was doing the job.
But I suppose kill -9 PID is not the way to go, or?
Core of my shutdown hook is below. The small comment - it is not important if I
have System.exit(0) or not, as the procedure never comes to this point :-).
Runtime.getRuntime().addShutdownHook( new Thread( () -> {
LOG.info( "Got shutdown requests ..." );
synchronized ( this ) {
streams.close( 5L, TimeUnit.SECONDS );
}
LOG.info( "Streamer is stopped." );
System.exit( 0 );
}
) );
> KStream close( withTimeout ) - does not work under load conditions in the
> multi-threaded KStream application
> ------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-5861
> URL: https://issues.apache.org/jira/browse/KAFKA-5861
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.11.0.0
> Reporter: Seweryn Habdank-Wojewodzki
>
> Recently implemented close( withTimeout ) for streams does not work under
> load conditions in the multi-threaded KStream application.
> Where there are more consuming threads and there many messages in stream,
> then close ( withTimeout ) does not work.
> 1. Timeout is not respected at all and
> 2. application is hanging in some streaming chaos. Theoretically threads are
> working - they are busy with themselves, so the app cannot end, but they are
> not processing any further messages.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
