NEWSFLASH: Wimpy latent scheduler buglet turns axe-murderer...
Root cause being us staying in kernel for weeks at a time during boot:
Subverting ftrace, we see below that updates _can_ be skipped for AGES,
which can make very bad things happen. It's supposed to skip ONE, as an
optimization. not lord knows how many. Here we see we've been skipping
since 4915630446, resulting in the watchdog being credited with a 137ms
delta_exec. Bogus, and..
modprobe-134 [000] 5.027903: update_rq_clock <-scheduler_tick
modprobe-134 [000] 5.037857: update_rq_clock <-scheduler_tick
modprobe-134 [000] 5.037864: update_rq_clock <-enqueue_task
modprobe-134 [000] 5.037866: enqueue_task_watchdog: WATCHDOG/0
enqueue 4915630446
modprobe-134 [000] 5.044965: update_rq_clock <-scheduler_tick
modprobe-134 [000] 5.046275: update_rq_clock <-scheduler_tick
modprobe-134 [000] 5.050274: update_rq_clock <-scheduler_tick
modprobe-134 [000] 5.052862: update_rq_clock <-dequeue_task
modprobe-134 [000] 5.052866: pick_next_task_watchdog: WATCHDOG/0
select 4915630446
watchdog/0-7 [000] 5.052868: finish_task_switch: WATCHDOG/0 runs
5052867644
watchdog/0-7 [000] 5.052869: watchdog: WATCHDOG/0 DELAYED
137238048 max: 137238048
watchdog/0-7 [000] 5.052872: update_rq_clock <-dequeue_task
watchdog/0-7 [000] 5.052873: dequeue_task_watchdog: WATCHDOG/0
dequeue 5052872018
watchdog/0-7 [000] 5.052874: put_prev_task_watchdog: WATCHDOG/0
STOP trace : delta_exec 137241572 sum_exec 137254287
..delta_exec was sufficient in a NOPREEMPT kernel on zillion disk box
that disk discovery combined with swsusp wanting to find out if it needs
to resume, that the featherweight watchdog was credited with delta_exec
HUGE enough to be incapacitated by the rt throttle long enough for the
hard lockup detector to trigger.
Throttled watchdog is bad juju, led to -ENOBOOT for IO beast.
Cc: <[email protected]>
Signed-off-by: Mike Galbraith <[email protected]>
---
kernel/sched/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -118,7 +118,7 @@ void update_rq_clock(struct rq *rq)
{
s64 delta;
- if (rq->skip_clock_update > 0)
+ if (rq->skip_clock_update-- > 0)
return;
delta = sched_clock_cpu(cpu_of(rq)) - rq->clock;
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
