https://gcc.gnu.org/bugzilla/show_bug.cgi?id=128231
Bug ID: 128231
Summary: Combine multiple individual horizontal reductions.
Product: gcc
Version: 17.0
URL: https://gcc.gnu.org/pipermail/gcc-patches/2026-June/72
1106.html
Status: UNCONFIRMED
Keywords: missed-optimization
Severity: normal
Priority: P3
Component: tree-optimization
Assignee: unassigned at gcc dot gnu.org
Reporter: dipesh.sharma at amd dot com
CC: dipesh.sharma at amd dot com, rguenth at gcc dot gnu.org,
venkataramanan.kumar at amd dot com
Target Milestone: ---
When a loop has several independent sums, the vectorizer keeps a separate
accumulator for each one. At the end of the loop each accumulator is turned
into a scalar with .REDUC_PLUS (IFN_REDUC_PLUS).
On a wide vector that reduction is a chain of cross-lane extract-and-add steps,
for example 512-bit down to 256, then 128, then 64, then 32. If the same block
has N such reductions, that chain is repeated N times. The cross-lane part is
the expensive one, and it does not have to be done separately for every sum.
Example:
int A[128], B0[128], B1[128], B2[128], B3[128];
int C[4];
void reduc (void)
{
int s0 = 0, s1 = 0, s2 = 0, s3 = 0;
for (int k = 0; k < 128; ++k)
{
int a = A[k];
s0 += a * B0[k];
s1 += a * B1[k];
s2 += a * B2[k];
s3 += a * B3[k];
}
C[0] = s0;
C[1] = s1;
C[2] = s2;
C[3] = s3;
}
Compile with -O3 -mavx512f. Each accumulator is a vector(16) int. The epilogue
has four separate reductions:
Gimple IR
_77 = .REDUC_PLUS (vect_s0);
_66 = .REDUC_PLUS (vect_s1);
_10 = .REDUC_PLUS (vect_s2);
_35 = .REDUC_PLUS (vect_s3)
ASM
vextracti32x8 $0x1, %zmm4, %ymm2
vpaddd %ymm4, %ymm2, %ymm2
vextracti32x4 $0x1, %ymm2, %xmm4
vpaddd %xmm2, %xmm4, %xmm2
vpshufd $238, %xmm2, %xmm4
vpaddd %xmm2, %xmm4, %xmm4
vpshufd $85, %xmm4, %xmm2
vpaddd %xmm4, %xmm2, %xmm2
vmovd %xmm2, C32+8(%rip)
... Repeated N times for each individual reduction.
The sequence can be replaced with one shared unpack/add tree, a single
narrowing path to 128 bits, and four scalar extracts:
e.g
vpunpckldq %zmm2, %zmm3, %zmm1
vpunpckhdq %zmm2, %zmm3, %zmm3
vpunpckldq %zmm7, %zmm0, %zmm2
vpunpckhdq %zmm7, %zmm0, %zmm0
vpaddd %zmm3, %zmm1, %zmm1
vpaddd %zmm0, %zmm2, %zmm2
vpunpckldq %zmm2, %zmm1, %zmm0
vpunpckhdq %zmm2, %zmm1, %zmm1
vpaddd %zmm1, %zmm0, %zmm0
vmovdqa %ymm0, %ymm1
vextracti64x4 $0x1, %zmm0, %ymm0
vpaddd %ymm0, %ymm1, %ymm1
vextracti128 $0x1, %ymm1, %xmm0
vpaddd %xmm1, %xmm0, %xmm0
vpshufd $141, %xmm0, %xmm1
vmovdqa %xmm1, C(%rip)
vpsrldq $8, %xmm0, %xmm1
vpaddd %xmm1, %xmm0, %xmm0
vpsrldq $4, %xmm0, %xmm1
vpaddd %xmm1, %xmm0, %xmm0
vmovd %xmm0, %eax
Latest patch to address this gap is posted and can be tracked here:
https://gcc.gnu.org/pipermail/gcc-patches/2026-October/736180.html
