For csky part.
Acked-by: Guo Ren
On Wed, May 25, 2022 at 7:45 AM Peter Xu wrote:
>
> I observed that for each of the shared file-backed page faults, we're very
> likely to retry one more time for the 1st write fault upon no page. It's
> because we'll need to release the mmap lock for dirty rate limit purpose
> with balance_dirty_pages_ratelimited() (in fault_dirty_shared_page()).
>
> Then after that throttling we return VM_FAULT_RETRY.
>
> We did that probably because VM_FAULT_RETRY is the only way we can return
> to the fault handler at that time telling it we've released the mmap lock.
>
> However that's not ideal because it's very likely the fault does not need
> to be retried at all since the pgtable was well installed before the
> throttling, so the next continuous fault (including taking mmap read lock,
> walk the pgtable, etc.) could be in most cases unnecessary.
>
> It's not only slowing down page faults for shared file-backed, but also add
> more mmap lock contention which is in most cases not needed at all.
>
> To observe this, one could try to write to some shmem page and look at
> "pgfault" value in /proc/vmstat, then we should expect 2 counts for each
> shmem write simply because we retried, and vm event "pgfault" will capture
> that.
>
> To make it more efficient, add a new VM_FAULT_COMPLETED return code just to
> show that we've completed the whole fault and released the lock. It's also
> a hint that we should very possibly not need another fault immediately on
> this page because we've just completed it.
>
> This patch provides a ~12% perf boost on my aarch64 test VM with a simple
> program sequentially dirtying 400MB shmem file being mmap()ed and these are
> the time it needs:
>
> Before: 650.980 ms (+-1.94%)
> After: 569.396 ms (+-1.38%)
>
> I believe it could help more than that.
>
> We need some special care on GUP and the s390 pgfault handler (for gmap
> code before returning from pgfault), the rest changes in the page fault
> handlers should be relatively straightforward.
>
> Another thing to mention is that mm_account_fault() does take this new
> fault as a generic fault to be accounted, unlike VM_FAULT_RETRY.
>
> I explicitly didn't touch hmm_vma_fault() and break_ksm() because they do
> not handle VM_FAULT_RETRY even with existing code, so I'm literally keeping
> them as-is.
>
> Signed-off-by: Peter Xu
> ---
>
> v3:
> - Rebase to akpm/mm-unstable
> - Copy arch maintainers
> ---
> arch/alpha/mm/fault.c | 4
> arch/arc/mm/fault.c | 4
> arch/arm/mm/fault.c | 4
> arch/arm64/mm/fault.c | 4
> arch/csky/mm/fault.c | 4
> arch/hexagon/mm/vm_fault.c| 4
> arch/ia64/mm/fault.c | 4
> arch/m68k/mm/fault.c | 4
> arch/microblaze/mm/fault.c| 4
> arch/mips/mm/fault.c | 4
> arch/nios2/mm/fault.c | 4
> arch/openrisc/mm/fault.c | 4
> arch/parisc/mm/fault.c| 4
> arch/powerpc/mm/copro_fault.c | 5 +
> arch/powerpc/mm/fault.c | 5 +
> arch/riscv/mm/fault.c | 4
> arch/s390/mm/fault.c | 12 +++-
> arch/sh/mm/fault.c| 4
> arch/sparc/mm/fault_32.c | 4
> arch/sparc/mm/fault_64.c | 5 +
> arch/um/kernel/trap.c | 4
> arch/x86/mm/fault.c | 4
> arch/xtensa/mm/fault.c| 4
> include/linux/mm_types.h | 2 ++
> mm/gup.c | 34 +-
> mm/memory.c | 2 +-
> 26 files changed, 138 insertions(+), 3 deletions(-)
>
> diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c
> index ec20c1004abf..ef427a6bdd1a 100644
> --- a/arch/alpha/mm/fault.c
> +++ b/arch/alpha/mm/fault.c
> @@ -155,6 +155,10 @@ do_page_fault(unsigned long address, unsigned long mmcsr,
> if (fault_signal_pending(fault, regs))
> return;
>
> + /* The fault is fully completed (including releasing mmap lock) */
> + if (fault & VM_FAULT_COMPLETED)
> + return;
> +
> if (unlikely(fault & VM_FAULT_ERROR)) {
> if (fault & VM_FAULT_OOM)
> goto out_of_memory;
> diff --git a/arch/arc/mm/fault.c b/arch/arc/mm/fault.c
> index dad27e4d69ff..5ca59a482632 100644
> --- a/arch/arc/mm/fault.c
> +++ b/arch/arc/mm/fault.c
> @@ -146,6 +146,10 @@ void do_page_fault(unsigned long address, struct pt_regs
> *regs)
> return;
> }
>
> + /* The fault is fully completed (including releasing mmap lock) */
> + if (fault & VM_FAULT_COMPLETED)
> + return;
> +
> /*
> * Fault retry nuances, mmap_lock already relinquished by core mm
> */
> diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
> index a062e07516dd..46cccd6bf705 100644
> --- a/arch/arm/mm/fault.c
> +++ b/arch/arm/mm/fault.c
> @@ -322,6 +322,10 @@ do_p
