Jira (PUP-5431) Puppet restarts services at shutdown

[Ioannis (JIRA)]

Title: Message Title

Ioannis created an issue   Puppet / PUP-5431 Puppet restarts services at shutdown Issue Type: Bug Assignee: Kylo Ginsberg Components: Client Created: 2015/10/27 6:22 AM Environment: $ uname -a Linux node1 2.6.32-504.el6.x86_64 #1 SMP Tue Sep 16 01:56:35 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a LSB Version: :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch Distributor ID: RedHatEnterpriseServer Description: Red Hat Enterprise Linux Server release 6.6 (Santiago) Release: 6.6 Codename: Santiago $ puppet --version 3.3.2 Priority: Major Reporter: Ioannis When the node is shutting down, puppet daemon is one of the first services being stopped. This make sense since puppet ensures other services are running. But the daemon process is forking another puppet agent process to apply the catalog and the initscript is only stopping the parent daemon. When the daemon periodically starts the agent, it can happens that during system shutdown, the daemon is stopped but the child continues to apply the catalog. It's easy to reproduce by terminating the service while a catalog is being applied. Oct 27 13:01:32 node2 puppet-agent[30677]: Starting Puppet client version 3.3.2 Oct 27 13:01:38 node2 puppet-agent[30677]: Caught TERM; calling stop Oct 27 13:02:03 node2 puppet-agent[30680]: Finished catalog run in 16.43 seconds So while services are being shutdown, puppet brings them up again to be killed ungracefully again by init. This has major implications when there's an clustering software running. While services have been properly de-registered from the cluster, puppet restarts services which then are marked as in a faulty state for the cluster. A quick fix could be to update the initscript to also terminate the child process. diff -Nur a/puppet b/puppet --- a/puppet 2015-10-27 10:35:54.011661982 +0000 +++ b/puppet 2015-10-27 10:37:05.601287405 +0000 @@ -50,10 +50,21 @@ stop() { echo -n $"Stopping puppet agent: " + # Get the daemon pid if exists + [[ -f $pidfile ]] && daemonpid=$(cat $pidfile) killproc $pidopts $puppetd RETVAL=$? echo - [ $RETVAL = 0 ] && rm -f ${lockfile} ${pidfile} + if [ $RETVAL = 0 ]; then + # Daemon is dead, clean up child processes and lock files + if [[ -n $daemonpid ]]; then + pkill -TERM -P $daemonpid || : + sleep 1 # grace period + pkill -KILL -P $daemonpid || : + fi + rm -f ${lockfile} ${pidfile} + fi + } Add Comment   This message was sent by Atlassian JIRA (v6.4.11#64026-sha1:78f6ec4) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.