In order to allow efficient implementation of GCS implementations are permitted to do GCS specific caching, with barriers implicit in stack switch operations and GCSB DSYNC as an explicit barrier. Since we do not use GCS in the hypervisor or host kernel the hypervisor cannot rely on any barriers being present in vCPU context switches other than those it explicitly inserts. Add explicit barriers when loading and saving vCPU state.
Signed-off-by: Mark Brown <[email protected]> Reviewed-by: Yuan Yao <[email protected]> --- arch/arm64/kvm/hyp/nvhe/switch.c | 13 +++++++++++++ arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 14 ++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 7318e3e6a5f3..fb4c9988c84d 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -290,6 +290,12 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) */ __debug_save_host_buffers_nvhe(vcpu); + /* + * Ensure any GCS memory effects are visible to this CPU. + */ + if (ctxt_has_gcs(guest_ctxt)) + gcsb_dsync(); + /* * We're about to restore some new MMU state. Make sure * ongoing page-table walks that have started before we @@ -338,6 +344,13 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __timer_disable_traps(vcpu); __hyp_vgic_save_state(vcpu); + /* + * Ensure any GCS memory effects from the outgoing vCPU are + * visible elsewhere. + */ + if (ctxt_has_gcs(guest_ctxt)) + gcsb_dsync(); + /* * Same thing as before the guest run: we're about to switch * the MMU context, so let's make sure we don't have any diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c index 4f775545c71b..6665d97eac95 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -228,6 +228,13 @@ void __vcpu_load_switch_sysregs(struct kvm_vcpu *vcpu) host_ctxt = host_data_ptr(host_ctxt); __sysreg_save_user_state(host_ctxt); + /* + * Ensure any GCS memory effects are visible to the incoming + * vCPU. + */ + if (ctxt_has_gcs(guest_ctxt)) + gcsb_dsync(); + /* * When running a normal EL1 guest, we only load a new vcpu * after a context switch, which involves a DSB, so all @@ -296,6 +303,13 @@ void __vcpu_put_switch_sysregs(struct kvm_vcpu *vcpu) __sysreg_save_user_state(guest_ctxt); __sysreg32_save_state(vcpu); + /* + * Ensure any GCS memory effects from the outgoing vCPU are + * visible elsewhere. + */ + if (ctxt_has_gcs(guest_ctxt)) + gcsb_dsync(); + /* Restore host user state */ __sysreg_restore_user_state(host_ctxt); -- 2.47.3
