On Tue, Jun 24, 2025 at 2:11 PM H.J. Lu <hjl.to...@gmail.com> wrote:
>
> On Mon, Jun 23, 2025 at 2:24 PM H.J. Lu <hjl.to...@gmail.com> wrote:
> >
> > On Wed, Jun 18, 2025 at 3:17 PM H.J. Lu <hjl.to...@gmail.com> wrote:
> > >
> > > 1. Don't generate the loop if the loop count is 1.
> > > 2. For memset with vector on small size, use vector if small size supports
> > > vector, otherwise use the scalar value.
> > > 3. Duplicate the promoted scalar value for vector.
> > > 4. Always expand vector-version of memset for vector_loop.
> > > 5. Use misaligned prologue if alignment isn't needed. When misaligned
> > > prologue is used, check if destination is actually aligned and update
> > > destination alignment if aligned.
> > >
> > > The included tests show that codegen of vector_loop/unrolled_loop for
> > > memset/memcpy are significantly improved. For
> > >
> > > ---
> > > void
> > > foo (void *p1, size_t len)
> > > {
> > > __builtin_memset (p1, 0, len);
> > > }
> > > ---
> > >
> > > with
> > >
> > > -O2 -minline-all-stringops
> > > -mmemset-strategy=vector_loop:256:noalign,libcall:-1:noalign -march=x86-64
> > >
> > > we used to generate
> > >
> > > foo:
> > > .LFB0:
> > > .cfi_startproc
> > > movq %rdi, %rax
> > > pxor %xmm0, %xmm0
> > > cmpq $64, %rsi
> > > jnb .L18
> > > .L2:
> > > andl $63, %esi
> > > je .L1
> > > xorl %edx, %edx
> > > testb $1, %sil
> > > je .L5
> > > movl $1, %edx
> > > movb $0, (%rax)
> > > cmpq %rsi, %rdx
> > > jnb .L19
> > > .L5:
> > > movb $0, (%rax,%rdx)
> > > movb $0, 1(%rax,%rdx)
> > > addq $2, %rdx
> > > cmpq %rsi, %rdx
> > > jb .L5
Lili found that the regression of 527.cam4_r (PR120943) is caused by
more instructions due to the usage of movb instruction(takes more
iterations) instead of original movq.
The patch optimizes it with vector moves and solves the issue.
> > > .L1:
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L18:
> > > movq %rsi, %rdx
> > > xorl %eax, %eax
> > > andq $-64, %rdx
> > > .L3:
> > > movups %xmm0, (%rdi,%rax)
> > > movups %xmm0, 16(%rdi,%rax)
> > > movups %xmm0, 32(%rdi,%rax)
> > > movups %xmm0, 48(%rdi,%rax)
> > > addq $64, %rax
> > > cmpq %rdx, %rax
> > > jb .L3
> > > addq %rdi, %rax
> > > jmp .L2
> > > .L19:
> > > ret
> > > .cfi_endproc
> > >
> > > with very poor prologue/epilogue. With this patch, we now generate:
> > >
> > > foo:
> > > .LFB0:
> > > .cfi_startproc
> > > pxor %xmm0, %xmm0
> > > cmpq $64, %rsi
> > > jnb .L2
> > > testb $32, %sil
> > > jne .L19
> > > testb $16, %sil
> > > jne .L20
> > > testb $8, %sil
> > > jne .L21
> > > testb $4, %sil
> > > jne .L22
> > > testq %rsi, %rsi
> > > jne .L23
> > > .L1:
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L2:
> > > movups %xmm0, -64(%rdi,%rsi)
> > > movups %xmm0, -48(%rdi,%rsi)
> > > movups %xmm0, -32(%rdi,%rsi)
> > > movups %xmm0, -16(%rdi,%rsi)
> > > subq $1, %rsi
> > > cmpq $64, %rsi
> > > jb .L1
> > > andq $-64, %rsi
> > > xorl %eax, %eax
> > > .L9:
> > > movups %xmm0, (%rdi,%rax)
> > > movups %xmm0, 16(%rdi,%rax)
> > > movups %xmm0, 32(%rdi,%rax)
> > > movups %xmm0, 48(%rdi,%rax)
> > > addq $64, %rax
> > > cmpq %rsi, %rax
> > > jb .L9
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L23:
> > > movb $0, (%rdi)
> > > testb $2, %sil
> > > je .L1
> > > xorl %eax, %eax
> > > movw %ax, -2(%rdi,%rsi)
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L19:
> > > movups %xmm0, (%rdi)
> > > movups %xmm0, 16(%rdi)
> > > movups %xmm0, -32(%rdi,%rsi)
> > > movups %xmm0, -16(%rdi,%rsi)
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L20:
> > > movups %xmm0, (%rdi)
> > > movups %xmm0, -16(%rdi,%rsi)
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L21:
> > > movq $0, (%rdi)
> > > movq $0, -8(%rdi,%rsi)
> > > ret
> > > .p2align 4,,10
> > > .p2align 3
> > > .L22:
> > > movl $0, (%rdi)
> > > movl $0, -4(%rdi,%rsi)
> > > ret
> > > .cfi_endproc
> >
> >
> > Here is the v2 patch with the memset improvements:
> >
> > 1. Always duplicate the promoted scalar value for vector_loop if not 0
> > nor -1.
> > 2. Update setmem_epilogue_gen_val to use the RTL info from the previous
> > iteration.
> >
> > OK for master?
> >
>
> Here is the v3 patch rebased against
>
> commit d073bb6cfc219d4b6c283a0b527ee88b42e640e0
> Author: H.J. Lu <hjl.to...@gmail.com>
> Date: Thu Mar 18 18:43:10 2021 -0700
>
> x86: Update memcpy/memset inline strategies for -mtune=generic
>
> OK for master?
Ok for trunk.
>
> Thanks.
>
> --
> H.J.
--
BR,
Hongtao
