From: Kyrylo Tkachov <[email protected]> early_ra::allocate_colors marks the FPRs occupied by a color with
m_allocated_fprs |= ((1U << color->group->size) - 1) << best; When a color group spans the whole register file (size == 32), as can happen for a heavily unrolled, vectorized loop, "1U << 32" is undefined and evaluates to 1 on AArch64 hosts, so the expression sets no bits at all. The 32 FPRs of the group are therefore not recorded as allocated. Subsequent colors (and broaden_colors) then reuse those registers, which breaks the invariant that distinct colors receive disjoint FPRs. In PR125795 this let the loop-invariant TBL permute index, which is live across the whole loop, share v28 with the LD2 tuple destinations, so the index was clobbered mid-loop and the loop produced wrong results. Fix this by using a 64-bit shift base: unsigned long long is at least 64 bits on every host, so "1ULL << 32" is well-defined. best + size <= 32 is guaranteed by the candidate search, which the patch also asserts, so the result still fits in the 32-bit m_allocated_fprs When the full-width group can no longer be hidden, allocate_colors correctly fails to find a register for the other color and the region is left to the real register allocator, matching -mearly-ra=none. Bootstrapped and tested on aarch64-none-linux-gnu. Pushing to trunk and later to the branches after testing. Signed-off-by: Kyrylo Tkachov <[email protected]> gcc/ChangeLog: PR target/125795 * config/aarch64/aarch64-early-ra.cc (early_ra::allocate_colors): Compute the allocated-FPR mask as ((1ULL << color->group->size) - 1) << best. gcc/testsuite/ChangeLog: PR target/125795 * gcc.target/aarch64/pr125795.c: New test. --- gcc/config/aarch64/aarch64-early-ra.cc | 8 +++- gcc/testsuite/gcc.target/aarch64/pr125795.c | 47 +++++++++++++++++++++ 2 files changed, 54 insertions(+), 1 deletion(-) create mode 100644 gcc/testsuite/gcc.target/aarch64/pr125795.c diff --git a/gcc/config/aarch64/aarch64-early-ra.cc b/gcc/config/aarch64/aarch64-early-ra.cc index 24f0e1bdffb..6982847a796 100644 --- a/gcc/config/aarch64/aarch64-early-ra.cc +++ b/gcc/config/aarch64/aarch64-early-ra.cc @@ -3038,7 +3038,13 @@ early_ra::allocate_colors () if (dump_file && (dump_flags & TDF_DETAILS)) fprintf (dump_file, " Allocating [v%d:v%d] to color %d\n", best, best + color->group->size - 1, color->id); - m_allocated_fprs |= ((1U << color->group->size) - 1) << best; + // Mark the COLOR's FPRs as allocated. A full-width color can have + // size == 32, so shift a wide enough value: "1U << 32" is undefined, + // as is "1UL << 32" on hosts with 32-bit long. unsigned long long is + // at least 64 bits everywhere. best + size <= 32 (from the candidate + // search) keeps the result within the 32-bit mask. + gcc_assert (best + color->group->size <= 32); + m_allocated_fprs |= ((1ULL << color->group->size) - 1) << best; } } diff --git a/gcc/testsuite/gcc.target/aarch64/pr125795.c b/gcc/testsuite/gcc.target/aarch64/pr125795.c new file mode 100644 index 00000000000..9f5c6a32b5d --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/pr125795.c @@ -0,0 +1,47 @@ +/* PR target/125795 */ +/* { dg-do run } */ +/* { dg-options "-O2 -funroll-loops" } */ + +/* This loop is vectorized with LD2 (deinterleave) and TBL (lane reversal), + then unrolled. The unrolled body builds a full-width (32 FPR) early-ra + color group, which used to trigger a "1U << 32" undefined shift in + early_ra::allocate_colors. As a result the long-lived TBL permute index + was wrongly allocated to the same FPR as an LD2 tuple destination and got + clobbered, so the result was miscomputed. */ + +extern void abort (void); + +#define N 256 + +static int __attribute__ ((noipa)) +foo (void) +{ + int k[N]; + int k2[N]; + int i; + + for (i = 0; i < N; i++) + k2[i] = i; + + for (i = 0; i < 64; i++) + { + k[i] = k2[2 * i + 0]; + k[255 - i] = k2[2 * i + 1]; + k[64 + i] = k2[2 * i + 128]; + k[191 - i] = k2[2 * i + 129]; + } + + for (i = 0; i < N; i++) + if (k[i] != ((i < 128) ? (2 * i) : (511 - 2 * i))) + return 1; + + return 0; +} + +int +main (void) +{ + if (foo ()) + abort (); + return 0; +} -- 2.50.1 (Apple Git-155)
