The problem, per commit fc98c3c8c9dc ("printk: use rcuidle console
tracepoint"), was printk usage from the cpuidle path where RCU was
already disabled.
Per the patches earlier in this series, this is no longer the case.
Signed-off-by: Peter Zijlstra (Intel)
---
kernel/printk/printk.c |2 +-
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-imx/cpuidle-imx6sx.c |5 -
1 file changed, 4 insertions(+), 1 deletion(-)
--- a/arch/arm/mach-imx/cpuidle-imx6s
CONFIG_GENERIC_ENTRY disallows any and all tracing when RCU isn't
enabled.
XXX if s390 (the only other GENERIC_ENTRY user as of this writing)
isn't comfortable with this, we could switch to
HAVE_NOINSTR_VALIDATION which is x86_64 only atm.
Signed-off-by: Peter Zijlstra (Intel)
---
include/linux
Ever since commit d3afc7f12987 ("arm64: Allow IPIs to be handled as
normal interrupts") this function is called in regular IRQ context.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm64/kernel/smp.c |4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/arch/arm64/kernel/smp.c
++
Doing RCU-idle outside the driver, only to then temporarily enable it
again before going idle is daft.
Notably: this converts all dt_init_idle_driver() and
__CPU_PM_CPU_IDLE_ENTER() users for they are inextrably intertwined.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-omap2/cpuidle3
No callers left that have already disabled RCU.
Signed-off-by: Peter Zijlstra (Intel)
---
kernel/time/tick-broadcast-hrtimer.c | 29 -
1 file changed, 12 insertions(+), 17 deletions(-)
--- a/kernel/time/tick-broadcast-hrtimer.c
+++ b/kernel/time/tick-broadcast-hrti
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/cpuidle-riscv-sbi.c |9 +
1 file changed, 5 insertions(+), 4 deletions(-)
--- a/drivers/cpuidle/cpuidle-r
Idle code is very like entry code in that RCU isn't available. As
such, add a little validation.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/alpha/kernel/vmlinux.lds.S |1 -
arch/arc/kernel/vmlinux.lds.S|1 -
arch/arm/include/asm/vmlinux.lds.h |1 -
arch/arm64/kerne
The __cpuidle functions will become a noinstr class, as such they need
explicit annotations.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/poll_state.c |6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
--- a/drivers/cpuidle/poll_state.c
+++ b/drivers/cpuidle/poll_state.c
All the idle routines are called with RCU disabled, as such there must
not be any tracing inside.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/acpi/processor_idle.c | 24 +---
1 file changed, 13 insertions(+), 11 deletions(-)
--- a/drivers/acpi/processor_idle.c
+++ b/
vmlinux.o: warning: objtool: intel_idle_s2idle+0x6e: call to
__monitor.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_irq+0x8c: call to
__monitor.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle+0x73: call to __monitor.constpro
The whole disable-RCU, enable-IRQS dance is very intricate since
changing IRQ state is traced, which depends on RCU.
Add two helpers for the cpuidle case that mirror the entry code.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-imx/cpuidle-imx6q.c|4 +--
arch/arm/mach-imx/cpui
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/cpuidle-psci.c |9 +
1 file changed, 5 insertions(+), 4 deletions(-)
--- a/drivers/cpuidle/cpuidle-psci.c
OMAP was the one and only user.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/clk/clk.c |8
1 file changed, 4 insertions(+), 4 deletions(-)
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -978,12 +978,12 @@ static void clk_core_disable(struct clk_
if (--core->enable_
arch_cpu_idle() is a very simple idle interface and exposes only a
single idle state and is expected to not require RCU and not do any
tracing/instrumentation.
As such, omap_sram_idle() is not a valid implementation. Replace it
with the simple (shallow) omap3_do_wfi() call. Leaving the more
compli
OMAP3 uses full SoC suspend modes as idle states, as such it needs the
whole power-domain and clock-domain code from the idle path.
All that code is not suitable to run with RCU disabled, as such push
RCU-idle deeper still.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-omap2/cpuidle34
Commit c227233ad64c ("intel_idle: enable interrupts before C1 on
Xeons") wrecked intel_idle in two ways:
- must not have tracing in idle functions
- must return with IRQs disabled
Additionally, it added a branch for no good reason.
Fixes: c227233ad64c ("intel_idle: enable interrupts before C1
Current arch_cpu_idle() is called with IRQs disabled, but will return
with IRQs enabled.
However, the very first thing the generic code does after calling
arch_cpu_idle() is raw_local_irq_disable(). This means that
architectures that can idle with IRQs disabled end up doing a
pointless 'enable-dis
All callers should still have RCU enabled.
Signed-off-by: Peter Zijlstra (Intel)
---
kernel/cpu_pm.c |9 -
1 file changed, 9 deletions(-)
--- a/kernel/cpu_pm.c
+++ b/kernel/cpu_pm.c
@@ -30,16 +30,9 @@ static int cpu_pm_notify(enum cpu_pm_eve
{
int ret;
- /*
-
Typical boot time setup; no need to suffer an indirect call for that.
Signed-off-by: Peter Zijlstra (Intel)
Reviewed-by: Frederic Weisbecker
---
arch/x86/kernel/process.c | 50 +-
1 file changed, 28 insertions(+), 22 deletions(-)
--- a/arch/x86/ker
Now that arch_cpu_idle() is expected to return with IRQs disabled,
avoid the useless STI/CLI dance.
Per the specs this is supposed to work, but nobody has yet relied up
this behaviour so broken implementations are possible.
Cc: Isaku Yamahata
Cc: kirill.shute...@linux.intel.com
Signed-off-by: Pe
vmlinux.o: warning: objtool: mwait_idle+0x5: call to
current_set_polling_and_test() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xc5: call to
current_set_polling_and_test() leaves .noinstr.text section
vmlinux.o: warning: objtool: cpu_idle_poll.isra.0
None of these functions should ever be ran with RCU disabled anymore.
Specifically, do_handle_IPI() is only called from handle_IPI() which
explicitly does irq_enter()/irq_exit() which ensures RCU is watching.
The problem with smp_cross_call() was, per commit 7c64cc0531fa ("arm: Use
_rcuidle for s
Doing RCU-idle outside the driver, only to then teporarily enable it
again before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-omap2/cpuidle34xx.c | 16
1 file changed, 16 insertions(+)
--- a/arch/arm/mach-omap2/cpuidle34xx.c
+++ b/arch/arm/mach
vmlinux.o: warning: objtool: io_idle+0xc: call to __inb.isra.0() leaves
.noinstr.text section
vmlinux.o: warning: objtool: acpi_idle_enter+0xfe: call to num_online_cpus()
leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_idle_enter+0x115: call to
acpi_idle_fallback_to_c1.isra.0() le
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/cpuidle-tegra.c | 21 -
1 file changed, 16 insertions(+), 5 deletions(-)
--- a/drivers/cpuidle/
Doing RCU-idle outside the driver, only to then temporarily enable it
again before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/cpuidle-mvebu-v7.c |7 +++
1 file changed, 7 insertions(+)
--- a/drivers/cpuidle/cpuidle-mvebu-v7.c
+++ b/drivers/cpuidle/cpui
Make cpuidle_enter_state() consistent with the s2idle variant and
verify ->enter() always returns with interrupts disabled.
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/cpuidle.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
--- a/drivers/cpuidle/cpuidle.c
+++
On Wed, Jun 8, 2022 at 4:47 PM Peter Zijlstra wrote:
>
> Commit c227233ad64c ("intel_idle: enable interrupts before C1 on
> Xeons") wrecked intel_idle in two ways:
>
> - must not have tracing in idle functions
> - must return with IRQs disabled
>
> Additionally, it added a branch for no good rea
On Wed, Jun 08, 2022 at 04:27:57PM +0200, Peter Zijlstra wrote:
> @@ -254,11 +255,18 @@ void omap_sram_idle(void)
>*/
> if (save_state)
> omap34xx_save_context(omap3_arm_context);
> +
> + if (rcuidle)
> + cpuidle_rcu_enter();
> +
> if (save_state ==
OMAP was the one and only user.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-omap2/powerdomain.c | 10 +-
drivers/base/power/runtime.c | 24
2 files changed, 17 insertions(+), 17 deletions(-)
--- a/arch/arm/mach-omap2/powerdomain.c
+++ b/arch
vmlinux.o: warning: objtool: __halt+0x2c: call to hcall_func.constprop.0()
leaves .noinstr.text section
vmlinux.o: warning: objtool: __halt+0x3f: call to __tdx_hypercall() leaves
.noinstr.text section
vmlinux.o: warning: objtool: __tdx_hypercall+0x66: call to
__tdx_hypercall_failed() leaves .noi
The perf_lopwr_cb() is called from the idle routines; there is no RCU
there, we must not enter tracing.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/x86/events/amd/brs.c | 13 +
arch/x86/include/asm/perf_event.h |2 +-
2 files changed, 6 insertions(+), 9 deletions(-)
vmlinux.o: warning: objtool: mwait_idle+0x47: call to
mds_idle_clear_cpu_buffers() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xa2: call to
mds_idle_clear_cpu_buffers() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle+0x91: call t
Hi All! (omg so many)
These here few patches mostly clear out the utter mess that is cpuidle vs
rcuidle.
At the end of the ride there's only 2 real RCU_NONIDLE() users left
arch/arm64/kernel/suspend.c:RCU_NONIDLE(__cpu_suspend_exit());
drivers/perf/arm_pmu.c: RCU
Doing RCU-idle outside the driver, only to then temporarily enable it
again, some *four* times, before going idle is daft.
Signed-off-by: Peter Zijlstra (Intel)
---
arch/arm/mach-omap2/cpuidle44xx.c | 29 ++---
1 file changed, 18 insertions(+), 11 deletions(-)
--- a/ar
cpuidle_state::enter() methods should be IRQ invariant
Signed-off-by: Peter Zijlstra (Intel)
---
drivers/cpuidle/poll_state.c |4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
--- a/drivers/cpuidle/poll_state.c
+++ b/drivers/cpuidle/poll_state.c
@@ -17,7 +17,7 @@ static int __cpuidle
vmlinux.o: warning: objtool: acpi_idle_enter_s2idle+0xde: call to wbinvd()
leaves .noinstr.text section
vmlinux.o: warning: objtool: default_idle+0x4: call to arch_safe_halt() leaves
.noinstr.text section
vmlinux.o: warning: objtool: xen_safe_halt+0xa: call to
HYPERVISOR_sched_op.constprop.0() l
Current rcu_idle_exit() is terminally broken because it uses
local_irq_{save,restore}(), which are traced which uses RCU.
However, now that all the callers are sure to have IRQs disabled, we
can remove these calls.
Signed-off-by: Peter Zijlstra (Intel)
Acked-by: Paul E. McKenney
---
kernel/rcu
On Wed, Jun 08, 2022 at 05:01:05PM +0200, Rafael J. Wysocki wrote:
> On Wed, Jun 8, 2022 at 4:47 PM Peter Zijlstra wrote:
> >
> > Commit c227233ad64c ("intel_idle: enable interrupts before C1 on
> > Xeons") wrecked intel_idle in two ways:
> >
> > - must not have tracing in idle functions
> > - m
On Wed, Jun 8, 2022 at 5:48 PM Peter Zijlstra wrote:
>
> On Wed, Jun 08, 2022 at 05:01:05PM +0200, Rafael J. Wysocki wrote:
> > On Wed, Jun 8, 2022 at 4:47 PM Peter Zijlstra wrote:
> > >
> > > Commit c227233ad64c ("intel_idle: enable interrupts before C1 on
> > > Xeons") wrecked intel_idle in two
On Wed, Jun 8, 2022 at 4:27 PM Peter Zijlstra wrote:
>
> arch_cpu_idle() is a very simple idle interface and exposes only a
> single idle state and is expected to not require RCU and not do any
> tracing/instrumentation.
>
> As such, omap_sram_idle() is not a valid implementation. Replace it
> wit
On Wed, Jun 8, 2022 at 4:27 PM Peter Zijlstra wrote:
>
> Current arch_cpu_idle() is called with IRQs disabled, but will return
> with IRQs enabled.
>
> However, the very first thing the generic code does after calling
> arch_cpu_idle() is raw_local_irq_disable(). This means that
> architectures th
On Wed, Jun 8, 2022 at 4:47 PM Peter Zijlstra wrote:
>
> Typical boot time setup; no need to suffer an indirect call for that.
>
> Signed-off-by: Peter Zijlstra (Intel)
> Reviewed-by: Frederic Weisbecker
Reviewed-by: Rafael J. Wysocki
> ---
> arch/x86/kernel/process.c | 50
> +
tree: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
master
head: 6bfb56e93bcef41859c2d5ab234ffd80b691be35
commit: e188f3330a13df904d77003846eafd3edf99009d ARC: cmpxchg/xchg: rewrite as
macros to make type safe
date: 10 months ago
config: arc-randconfig-s032-20220530
(h
45 matches
Mail list logo
