(I have cloned the bug for this, to keep this particular
sub-discussion separable.)
As I have reported, we have a problem with non-forking daemon
readiness protocols.
There are two competing protocols that I'm aware of (not counting the
protocol implied by daemon(3)): upstart's, where the daemon raises
SIGSTOP, and systemd's where the daemon connects to an AF_UNIX socket
(whose name is found in an environment variable) and uses sendmsg to
send a message with SCM_CREDENTIALS.
(I checked djb's daemontools and it doesn't seem to address this
problem. If there is another existing approach please would someone
let me know.)
The systemd protocol is troublesome because it requires too much code
in the daemon, leaving the daemon author with the trilemma which has
previously been discussed. The upstart protocol is "elegant" or
"horrible" depending who you listen to; and has the minor practical
problem that it can make sending stop signals to starting daemons
ineffective or damaging.
The upstart protocol could be somewhat improved by the use of SIGTTOU
rather than SIGSTOP, but the result still doesn't provide 100%
reliability for an administrator's use of SIGSTOP (a SIGSTOP sent
between the daemon raising TTOU and being sent CONT by the supervisor
will be lost). And anyway that doesn't answer the key aesthetic
objection that signalling readiness by stopping is perverse. (I
certainly do not discount this objection simply because it's
aesthetic, even though as it happens I don't share it.)
I conclude therefore that we should design another simple protocol -
preferably, a variation on one of the existing ones - and have (at
least) both Debian's systemd and Debian's upstart implement it.
Obviously this needs input from both upstreams. Given the poor
relationship between the two main projects which would need to
implement the same protocol, and the possibility that we might have to
carry this in a local patch in Debian if we can't reach agreement with
one or other of the upstreams, I think it would be best if this design
discussion were refereed by the TC.
Also, though should not block the decision on the default init system
on this more open-ended design work (and associated negotation); but
it is probably worth waiting with starting a mass package conversion
until we know what startup protocol we want.
Onto the concrete question, it seems to me that the requirements for a
protocol that everyone should be able to accept include:
- it should be implementable on any reasonable unix
- it should have no (not just minor) undesired side effects
- it should require minimal code in the daemon
- it should not introduce new dependencies
Furthermore, I would say that:
- the indication to the daemon that it is to use the new protocol
should be providable either by command-line option or environment
variable - at the option of the daemon author; if it is done by
environment variable this should happen only if needed by the
specific daemon, and ideally by an environment variable specific to
the target program (to reduce the chances of the daemon's
descendants seeing and trying to honour it);
- the choice of fd to use should not be "baked in" to the protocol,
to make the protocol the easiest fit with other software.
I think there is only one realistic possible basic structure for such
a protocol: the supervisor passes a fd to the daemon by inheritance;
when the daemon is ready it calls write(2) to write a fixed string to
the socket.
The systemd approach of using a SOCK_SEQPACKET socket is attractive,
but unfortunately I don't think it's suitable because: many people are
unfamiliar with SOCK_SEQPACKET sockets (and we want a protocol which
daemon authors will be confident that they have implemented
correctly); it is difficult to debug with ordinary utilities (so a
daemon author can't check their implementation); and I have heard that
some kernels have idiosyncracies in their handling of these sockets.
However, we don't need to extend this protocol to continue throughout
the life of the daemon - if we wanted the kind of facilities provided
by the systemd messaging protocol, the extra library dependency is
tolerable so we could just use the systemd protocol in those cases.
Our new protocol is just for the very basic case. So we can just
indicate the end of the message by closing the socket.
I therefore propose the following specification:
* This protocol is called "simple non-forking daemon readiness
indication protocol" or "readiness protocol" for short.
* A service supervisor which implements the readiness protocol:
- MUST permit its configuration to specify command line arguments
and environment variables to the daemon;
- MUST NOT pass any startup protocol related environment variables
unless explicitly configured to do so as just described;
- MUST permit its configuration to specify the use of the
readiness protocol
- The fd number to be used MUST be specifiable by the
configuration; if there is a default for the fd, it MUST
be fixed and SHOULD be 3.
When the readiness protocol is in use, the supervisor:
- MUST create a suitable IPC object (probably a socket or pipe)
and exec the daemon with that object open on the specified fd.
"Suitable" means one where the daemon's syscalls (as specified
below) will DTRT.
- MUST provide the daemon with writeable object(s) suitable for
logging, on both stdout and stderr. These object(s) MUST be
terminal(s), pipe(s), fifo(s) or AF_UNIX SOCK_STREAM sockets.
- MUST arrange that the readiness fd, and stdout and stderr, are
initially nonblocking, not CLOEXEC, and these MUST NOT generate
any signals when written or closed. However, the objects MAY
generate SIGPIPE if written to after a failure of the
supervisor;
- MUST expect the daemon to write and close the socket as
described below.
* A daemon which supports this protocol MUST provide and document
at least one of:
- Command line option(s) which enable the protocol, which SHOULD
be able to specify the fd number, and if there is a fixed or
default fd number, it MUST be documented and SHOULD be 3; or
- An environment variable name which if found in the daemon's
environment enables the protocol, and whose value is the fd to
use (in decimal).
If the readiness protocol is enabled, the daemon:
- MUST NOT fork into the background, setpgrp, setsid, etc.;
- MAY write log and error messages to stderr and stdout;
- MUST NOT read from stdin (although a daemon MAY provide a means
to be told it has permission to read from stdin);
- when it is ready and providing its services, MUST call
write(fd,"READY=1",7);
close(fd);
- if either of those calls fail it SHOULD fail, exiting nonzero;
- MUST NOT exit due to failure, whether before or after readiness,
without writing an appropriate message to stderr and/or its
defined logging arrangement;
- MUST NOT write anything different to the readiness fd;
- If the readiness protocol was enabled by an environment
variable, MUST remove the variable from the environment of any
general-purpose children;
- MUST NOT allow the readiness fd to be inherited by any
general-purpose children.
(A "general-purpose child" is one whose behaviour is not entirely
determined in advance, when the daemon code is written.)
This is very like the systemd readiness protocol, but:
- the daemon inherits the socket rather than having to create and
connect it;
- the daemon can use write(2) rather than sendmsg with
SCM_CREDENTIALS;
- the daemon closes the socket when it's done;
- how use of this protocol is requested from the daemon.
As I say I don't expect this to replace systemd's native readiness
protocol which has some more sophisticated features. My intention is
that both upstart and systemd would provide this as one of the startup
options and that it would be used for daemons which just want to give
a basic readiness indication.
I'd like to invite the Debian maintainers for both systemd and upstart
to comment on my observations and on this proposal, and to pass it to
their respective upstreams for comment here.
Thanks,
Ian.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
