On 08.02.2016 18:18, Yuriy M. Kaminskiy wrote:
On 08.02.2016 02:15, Yuriy M. Kaminskiy wrote:
Package: systemd
Version: 215-17+deb8u3
Severity: important

Probably related:
cron-update.service is triggered by some /etc/cron* directories change and invokes `systemctl daemon-reload` and `systemctl try-restart cron.target`. Maybe there are some racing when it is triggered right when cron.target is being stopped?

Probably related upstream commit: 96fb8242cc1ef6b0e28f6c86a4f57950095dd7f1 (aka v216-30-g96fb824), however, it likely fixes symptoms [assert() and abort], but not underlying issue [racing or whatever].

I've looked at core file, after musing a bit upon sources, I don't think this commit will fix/hide issue.

Backtrace:

#6  0x00007f08e081124f in service_enter_start (s=s@entry=0x7f08e21c7a10)
    at ../src/core/service.c:1312
#7 0x00007f08e0813341 in service_sigchld_event.lto_priv.377 (u=0x7f08e21c7a10,
    pid=<optimized out>, code=<optimized out>, status=0)
    at ../src/core/service.c:2338
#8  0x00007f08e084b887 in manager_dispatch_sigchld (m=0x7f08e20fc350)
    at ../src/core/manager.c:1639

(gdb) p s->type
$14 = _SERVICE_TYPE_INVALID
(gdb) p s->state
$15 = SERVICE_START_PRE
(gdb)  p s->meta.load_state
$16 = UNIT_NOT_FOUND
(gdb) p s->exec_command
$18 = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0}

Problem is, we started executing unit, spawned StartPre command, then unit file was removed, systemctl daemon-reload was issued, unit structure become half-ghost, then we got SIGCHLD for that StartPre command from the already-removed unit. Oops.

With 96fb824 applied, end result would be same:

@@ -1332,6 +1345,12 @@ static void service_enter_start(Service *s) {
c = s->main_command = s->exec_command[SERVICE_EXEC_START];
         }

+        if (!c) {
+                assert(s->type == SERVICE_ONESHOT);
+                service_enter_start_post(s);
+                return;
+        }
+

c is NULL, s->type here is _SERVICE_TYPE_INVALID, so we'll die in assert anyway :-\

It is possible that upstream systemd version is still affected, you may want to try install jessie's systemd-cron 1.3.* into sid and play with install/removal in a loop. Completely untested patches for systemd master and backport to v215 is attached.

FWIW, patch from previous message is runtime-tested in very minimal qemu/kvm guest and works to some extent (i.e. prevent crash, leaves expected error message about sigchld-to-ghost-unit; but likely there are some issues remaining, as ghost of "cron-update.service" remain lingering around, even after apt-get purge systemd-cron and systemctl daemon-re{load,exec}; but at least, it does not crash systemd anymore).

Reply via email to