On Wed, Aug 02, 2017 at 01:38:40PM +0200, Peter Zijlstra wrote: > Since its inception, our understanding of ACQUIRE, esp. as applied to > spinlocks, has changed somewhat. Also, I wonder if, with a simple > change, we cannot make it provide more. > > The problem with the comment is that the STORE done by spin_lock isn't > itself ordered by the ACQUIRE, and therefore a later LOAD can pass over > it and cross with any prior STORE, rendering the default WMB > insufficient (pointed out by Alan). > > Now, this is only really a problem on PowerPC and ARM64, both of > which already defined smp_mb__before_spinlock() as a smp_mb(). > > At the same time, we can get a much stronger construct if we place > that same barrier _inside_ the spin_lock(). In that case we upgrade > the RCpc spinlock to an RCsc. That would make all schedule() calls > fully transitive against one another. > > Cc: Alan Stern <[email protected]> > Cc: Nicholas Piggin <[email protected]> > Cc: Ingo Molnar <[email protected]> > Cc: Will Deacon <[email protected]> > Cc: Linus Torvalds <[email protected]> > Cc: Michael Ellerman <[email protected]> > Cc: Oleg Nesterov <[email protected]> > Cc: Benjamin Herrenschmidt <[email protected]> > Cc: Paul McKenney <[email protected]> > Signed-off-by: Peter Zijlstra (Intel) <[email protected]> > --- > arch/arm64/include/asm/spinlock.h | 2 ++ > arch/powerpc/include/asm/spinlock.h | 3 +++ > include/linux/atomic.h | 3 +++ > include/linux/spinlock.h | 36 > ++++++++++++++++++++++++++++++++++++ > kernel/sched/core.c | 4 ++-- > 5 files changed, 46 insertions(+), 2 deletions(-) > > --- a/arch/arm64/include/asm/spinlock.h > +++ b/arch/arm64/include/asm/spinlock.h > @@ -367,5 +367,7 @@ static inline int arch_read_trylock(arch > * smp_mb__before_spinlock() can restore the required ordering. > */ > #define smp_mb__before_spinlock() smp_mb() > +/* See include/linux/spinlock.h */ > +#define smp_mb__after_spinlock() smp_mb() > > #endif /* __ASM_SPINLOCK_H */
Acked-by: Will Deacon <[email protected]> Will
