Re: mm,console: circular dependency between console_sem and zone lock

[Peter Zijlstra]

On Sun, Jun 01, 2014 at 10:08:21AM -0400, Sasha Levin wrote:
> On 05/12/2014 12:28 PM, Jan Kara wrote:
> > On Wed 07-05-14 22:03:08, Sasha Levin wrote:
> >> > Hi all,
> >> > 
> >> > While fuzzing with trinity inside a KVM tools guest running the latest 
> >> > -next
> >> > kernel I've stumbled on the following spew:
> >   Thanks for report. So the problem seems to be maginally valid but I'm not
> > 100% sure whom to blame :). So printk() code calls up() which calls
> > try_to_wake_up() under console_sem.lock spinlock. That function can take
> > rq->lock which is all expected.
> > 
> > The next part of the chain is that during CPU initialization we call
> > __sched_fork() with rq->lock which calls into hrtimer_init() which can
> > allocate memory which creates a dependency rq->lock => zone.lock.rlock.
> > 
> > And memory management code calls printk() which zone.lock.rlock held which
> > closes the loop. Now I suspect the second link in the chain can happen only
> > while CPU is booting and might even happen only if some debug options are
> > enabled. But I don't really know scheduler code well enough. Steven?
> 
> I've cc'ed Peter and Ingo who may be able to answer that, as it still happens
> on -next.

Ah, cute.

So the second paragraph seems to miss the detail that this is the
__sched_fork() call from init_idle(), all other callers don't actually
hold the rq->lock.

Now init_idle() is called from:

        sched_init()
        fork_idle()
        idle_thread_get()

Now fork_idle() is called from:

        smp_init() -> idle_threads_init() -> idle_init()

and idle_thread_get is called from:

        _cpu_up()


So while it looks we're calling __sched_fork() twice for every !boot
idle thread (urgh) we do appear to call it before anything is running on
that cpu, so I don't see any particular problem with removing the call
from under that lock.

Something like so should do I suppose.

---
 kernel/sched/core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 240aa83e73f5..99609c33482b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4505,9 +4505,10 @@ void init_idle(struct task_struct *idle, int cpu)
        struct rq *rq = cpu_rq(cpu);
        unsigned long flags;
 
+       __sched_fork(0, idle);
+
        raw_spin_lock_irqsave(&rq->lock, flags);
 
-       __sched_fork(0, idle);
        idle->state = TASK_RUNNING;
        idle->se.exec_start = sched_clock();
 

pgpvoYmBtinG5.pgp
Description: PGP signature